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APPENDIX A 


Evaluation Methodology 


Appendix A 


EVALUATION METHODOLOGY 


This appendix describes the design and methodology that COSMOS Corporation and 
its subcontractor The McKenzie Group (COSMOS-TMG) followed during the first year of 
the CSRD field-focused study. 


Proposed Designed and Rationale 


Because the evaluation is investigating directly the potential stimulant or catalytic role 
of CSRD by examining more closely the implementation and educational processes 
occurring at a given CSRD school, the team decided to use the case study method (e.g., 
Yin, 1994 and 1998) as the primary research tool. Investigators assessing a directly 
comparable initiative—the scale-up phase of the New American Schools (NAS)—arrived 
at the same methodological conclusion, even though NAS funding levels were higher than 
CSRD funding. The investigators described their rationale as follows: 


“The intervention assessed here is highly complex and embedded in 
‘real’ schools in ‘real’ districts. The relationships are so involved and 
dynamic that they argue against an experimental design or comparison 
group approach. The interventions and research questions argue 
instead for a replicated case-study approach, with the unit of analysis 
being the implementing school” (Bodilly, 1998, p. 25). [Emphasis 
added. | 


However, merely selecting the case study method was insufficient to produce the 
required results. Specific features of the case study method—not necessarily present in the 
NAS study—were incorporated into the design and data collection plans from the outset of 
study. The case studies rely heavily on three methodological tactics: using a ‘theoretical’ 
framework (also known as a logic model) to track events within a school; carefully 
establishing chronological benchmarks in tracking events; and explicitly addressing rival 
hypotheses. 


Selecting Case Study Sites: CSRD Schools to be Studied 


Goal of Case Study Sample: To Support Both Statistical and Analytic 
Generalization. The sample was drawn in an attempt to satisfy both types of 
generalization. Statistical generalization aims at making the findings from a sample of 
datapoints generalizable to the universe of all datapoints. Analytic generalization aims at 
making the findings from a study generalizable to a broader theoretical framework. 


The goal of the site selection process was to select a representative sample of CSRD 


schools by stratifying the universe of schools. The parameters were selected from among 
the characteristics that the Southwest Educational Development Laboratory collects as part 
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of its CSRD database, including: school type (e.g., elementary, middle, or high); award 
cohort; type of geographic area; and region of country. The final result was a stratified 
random sample. 


Two-Stage Sampling Procedure. The desire to have a representative sample of cases 
called for a stratified random sample of all schools in the CSRD program. The desire to 
support analytic generalization called for attending to the identity of CSRD models to be 
studied. These dual needs led to a two-stage sampling procedure. 


The two stages emanated from the observation that the CSRD schools had adopted 
over 227 different educational models—far beyond the 17 originally identified by the 
congressional conferees in the CSRD legislation—by the time the universe was established 
in July 2000 (Exhibit A-1). Under these circumstances, drawing any random sample would 
not produce the needed pairing of schools implementing the same CSRD model. However, 
a critical feature of the distribution of the 227 CSRD models was that a much smaller 
number of the models (about 10 percent) involved nearly two-thirds of all the 1,799 
schools.’ In other words, a small number of CSRD models has been implemented by the 
vast majority of the schools, and a large number of the CSRD models has been 
implemented by only one, two, or a few schools.” 


These circumstances raised the possibility that the first stage of the sampling 
procedure might be to split the entire universe of schools into several portions: Exhibit A-2 
shows that a cutoff retaining 50 percent of the schools only involves 10 CSRD models; 
retaining 60 percent of the schools involves only 15 CSRD models; and retaining 67 
percent or two-thirds of all the schools involves 21 models. Exhibit A-3 then shows the 20 
school and geographic features at these different cutoffs—comparing the aggregate profile 
of the first (retained) portion with that of the second (cutoff) portion. 


‘All data are from the “Database of Schools Awarded CSRD Funds to Implement Comprehensive 
Reform Models” (SEDL database), as of July 2000. 


*Furthermore, except for the large number of schools involved with the first two models, the full 
arraying of all 227 models revealed a steadily declining histogram according to the number of schools per 
model, with no particular gaps, bumps, or irregularities. 
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Exhibit A-1 


TOP 25 MODELS ADOPTED BY SCHOOLS, 
COMPARED TO MODELS IN ORIGINAL CSRD CONGRESSIONAL LANGUAGE 
AND 8 NEW AMERICAN SCHOOL (NAS) DESIGNS 


(July 2000) 

Rank, According to Identified by Congress. Part of 

Model Name No. of Schools Adopted Conferees’ NAS" 

Success for All 252 Yes No" 
Accelerated Schools 123 Yes! No 
Lightspan 109 No No 
Direct Instruction 61 Yes No 
America’s Choice 60 No Yes 
Coalition of Essential Schools 53 Yes No 
High Schools that Work 52 Yes No 
Co-NECT 46 Yes Yes 
Core Knowledge 45 No No 
HOSTS 38 No No 
Effective Schools 36 No No 
Ventures Initiatives and Focus 33 Yes No 
School Development Program 32 Yes No 
Expeditionary Learning Outward Bound 29 Yes Yes 
Different Ways of Knowing 27 No No 
AVID 26 No No 
Community for Learning 25 Yes No 
Modern Red School House 25 Yes Yes 
Roots and Wings 23 Yes Yes 
DePaul Univ. Ctr for Urban Ed. 22 No No 
Middle Start 21 No No 
Reading Recovery 20 No No 
ATLAS Communities 19 Yes Yes 
Onward to Excellence IT 18 No No 
Reading Renaissance" 17 No No 


* The conferees also named the following four models that were not among the top 25: Audrey Cohen 

College, Paideia, Talent Development High School, and Urban Learning Centers. 
** NAS also named the following two models not among the top 25: Audrey Cohen College and Urban 

Learning Centers. 
*** NAS supported Roots and Wings and Success for All, but only explicitly identified the former in its 
original design. 

T Cited as “National Alliance for Restructuring Education” at the time of the conferees’ report. 

™ Two additional models, Carbo National Reading Styles and DePaul University School Achievement 
Structure, are used as the primary model in 17 schools; Reading Renaissance was included because it is also 
being used as a secondary model in 3 schools. 
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Exhibit A-2 


RANKING OF CSRD MODELS IN RELATION TO PERCENT OF SCHOOLS 
COVERED 


Rank, According to umulative # of/Cumulative % of 
# of Schools Adopted Model Name # of Schools Schools All Schools 


uccess for All 252 252 


ccelerated Schools 1 375 
109 484 
irect Instruction 61 545 
605 
Oalition of Essential Schools 658 
igh Schools that Work 710 
Co-NECT 756 
Core Knowledge 801 


OSTS 839 49.70 About 50%of 
ffective Schools 875 oe iA Schools 
entures Initiatives and Focus 908 

940 


chool Development Program 
57.41 
59.00 About 60% of 
1021 60.49 ICSRD Schools 


1044 61.85 
1064 63.03 


1083 64.16 
1101 65.23 


1118 66.23 About two-thirds 
1135 67.24 of CSRD Schools 
1150 68.13 

1165 69.02 


1180 69.91 
1195 70.79 


1209 71.62 


1222 72.39 
1234 73.10 
1245 73.76 


1256 74.41 
1267 75.06 


(Continued on next page) 
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Exhibit A-2 (Continued) 


Rank, According to Cumulative # | Cumulative % 
# of Schools Adopted Model Name # of Schools of Schools of All Schools 
National Writing Project 1278 


Strategic Teaching and Reading 1299 76.95 
Success-in-the-Making 1309 77.55 


Models Implemented at Fewer 3 1688 100.00 
than 10 schools 


Source: SEDL Database, July 2000. At that time, 1,799 schools were in the database, 115 were removed 
from this total, for the following reasons: 1) Community for Learning (29 schools), because such a large 
portion of them are in one area—Pennsylvania; 2) AVID (26 schools), because all are in only one area—Texas; 
3) DePaul University Center for Urban Education model (22 schools); 4) DePaul University School 
Achievement Structure (17 schools), because all schools are in Chicago; and 5) Middle Start (21 schools), 
because all the schools are in only one area—Michigan. Of the five models, only one (Community for 
Learning) appeared in the original congressional conferees’ list of 17 models. 


Talent Development High School = aa 1289 76.36 


11 
11 
10 
10 
79 


The comparison in Exhibit A-3 reveals minor differences in school or geographic 
profiles between the retained and cutoff portions. For example, even the smallest retention 
(50 percent) shows that the two portions differ by only a few percentage points on all the 
school and geographic features, with differences exceeding three percentage points for only 
three of the 20 features: more in the retained portion were in rural (as opposed to large 
central city) areas, and in the South (as opposed to the Midwest). (As the cutoff point is 
raised to the 60 and 67 percent levels, all differences between the retained portion and the 
total universe diminish.) In concluding the first stage of the sampling procedure, and to 
reduce the number of models eligible for study, the 50 percent cutoff point was therefore 
initially selected: All schools had to be involved with one of the top 10 CSRD models. 


When the original evaluation design was presented to the CSRD Technical Work 
Group in January 2000, members of the group suggested that a school’s involvement in the 
top 10 models, being national models, could be qualitatively different from its involvement 
with a locally developed model—i.e., one in which the school itself may be invested or 
involved in the development process and not just in the implementation process.’ Lessons 
about the schoolwide reform process could be substantially different under this latter 
condition. As a result, schools adopting the 10 original models were augmented by schools 
falling into an 11th category—“all locally developed models.” This category added 
another 106 candidate schools to the original number covered by the top 10 models. The 
total universe thus became 945. 


’Such involvement by the school will be used as the operational definition of a “locally developed” 
model. 
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Exhibit A-3 


AGGREGATE PROFILES OF SCHOOLS AT THREE CUTOFF POINTS 
(COMPARING RETAINED AND CUTOFF PORTIONS)'” 


No. of |__School Type’ | Cohort | award | Pov. | Title 1 Area Type’ 


£00Z ‘uonvsodio> SOWSOD 


9-V 


Schls. | E Size | Rate | School | SW 3/4/51] 6 
Subtotal Retained (50%) |839 {65 |11 {11 |13 ]29 |71 |$70,162 |72  |86 65 [23 |2 |22 |22 |12 |14 Jo |22 15 
Cutoff (50%) 849 |62 |14 |11 |13 |26 |74 |$68,780 |68 | 86 65 [28 |2 |23 |17 |10 |12 |7 |25 {15 
Total 1688 |63 |13 [11 |13 |28 |72 |$69,471 |70  |86 65 [25 |2 |22 |19 fai |13 Je |23 |15 
Subtotal Retained (60%) |996  |63 |11 {11 |15 ]28 |72 |$70,037 |72  |86 64 |25 |2 |22 |21 fai |a4 Jo f22 fas 
Cutoff (40%) 692 64 |15 |10 |11 |28 |72 |$68,647 |67 | 87 65 [26 |2 | 23/17 |11 |12 Ja |24 |16 
Total 1688 |63 |13 [11 [13 |28 |72 |$69,471 |70  |86 65 [25 [2 |22 |19 [a1 [13 f6 [23 {15 
Subtotal Retained (67%) a R is i bg Ls Ce er Ln ie ie L, ; Mi is ie hs . 2 .s 
Cutoff (33%) 570 63 |15 |11 [11 [30 |70 |$69,107 |68 {85 64 |28 |2 |24 [16 [a1 [a1 [a f25_ a7 
Total 1688 |63 |13 [11 [13 |28 |72 |$69,471 |70  |86 65 [25 |2 |22 |i9 fai |13 fe [23 Jas 


Note: ‘Excludes AVID (n=26). Models associated with DePaul University (n=39), Communities for Learning (n=29), Middle Start (n=21) and 
locally developed models (n=106). 


“All entries are percentages, unless otherwise noted. 


3E=Elementary; M=Middle, S=Secondary; and C=Combined grade levels. 


“1=Large center city; 2=Large town; 3=Mid-size center city; 4=Rural; 5=Small town; 6=Fringe of large city; 7=Fringe of small city. 


Source: SEDL Database, as of July 2000. 


Given the modified cutoff, the second stage of the procedure then called for defining a 
stratified random sample of the retained 50 percent of schools, augmented by the 106 
schools in the “locally developed” category. 


First, the sample was stratified so that two or more schools came from the same 
school district; in this way, the evaluation could determine the influence of contextual 
conditions involving district policies are practices. For instance, if two schools in the same 
district appear to have similar experiences even though they adopted different models, the 
likelihood is greater that district conditions had an influence over the process. Conversely, 
if two schools with the same model but in different districts appeared to have similar 
experiences, the likelihood is greater that the common model had an influence over the 
process. (The district limitation also reduced the costs of the study by reducing the travel 
and labor costs invested into the collection and analysis of district data.). This stratification 
reduced the universe from 945 to 569 schools. Exhibit A-4 contains the aggregate profiles 
of schools in this stratified sample, compared to schools in the top cutoff category (using 
the top ten models, or 50% of the universe), as well as the entire universe of schools. 


Exhibit A-4 reveals some additional differences in school or geographic profiles 
between the stratified sample and the first cutoff portion as well as the total universe. The 
sample contains: 


@ More elementary schools (and fewer combined grade schools); 
@ Schools with higher poverty rates; 


@ More schools in large and mid-size center cities and fewer in large 
and small towns and rural areas; and 


@ More schools in the Northeast (as compared to the West). 


From the stratified sample of 569 schools, a final random sample of 108 schools in 54 
districts were selected for screening. The schools were categorized according to 
characteristics like grade level, cohort, and model adopted.* The team sorted the schools 
into 9 groups of 12 schools from 6 districts, so that if a school or district was eliminated 
during the screening process, the next set of schools in the group could be used (rather than 
drawing anew sample). Exhibit A-4 shows the characteristics of the schools selected for 
screening. 


“The evaluation team encountered some difficulty in finding pairs of schools within the same district 
that were adopting different models, particularly without overrepresenting the Success for All model (the top 
model selected) and underrepresenting some of the less popular model. 
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Exhibit A-4 


AGGREGATE PROFILES OF SCHOOLS AT TWO SAMPLE LEVELS 
COMPARED TO SCHOOLS IN THE FIRST CUTOFF PORTIONS, 
AND COMPARED TO THE UNIVERSE AS A WHOLE” 


No. of |__School Type’ | Cohort | award | Pov. | Title 1 Area Type’ Region 

Sample Level Schls.| E|M|S |C} 1 | 2 Size Rate | School |SW| 1) 2|3)]4{]5 |6{|7{]MW),NE;S/W 
Sample Selected for Screening [108 52 |24 |17 |8 |35 |65 |$71,046 |73 86 76 145 |O /23 |16 |5 |10 |2 |19 21 =|47 |13 
Sample strat. so more than one {569 66 |11 {12 |12 |39 |61 |$69,785 |75 87 68 |36 jO )25 |14 77 [12 |5) 122 21 |45 |12 
schools is in each district 
Subtotal Retained (50%) + local [945 63 |11 {12 |14 |31 |69 |$70,405 |72 93 66 |23 |2 |23 {22 |11 |14 |5 23 16 |44 /16 
models 
Total 1688 |63 {13 |11 |13 |28 |72 |$69,471 |70 86 65 (25 |2 |/22 {19 |11 |13 |6 |23 15 |44 |17 


Note: ‘Excludes AVID (n=26). Models associated with DePaul University (n=39), Communities for Learning (n=29), Middle Start (n=21) and locally 
developed models (n=106). 
?All entries are percentages, unless otherwise noted. 
3E=Elementary; M=Middle, S=Secondary; and C=Combined grade levels. 
41=Large center city; 2=Large town; 3=Mid-size center city; 4=Rural; 5=Small town; 6=Fringe of large city; 7=Fringe of small city. 


Source: SEDL Database, as of July 2000. 


Screening Case Study Sites 


Once schools were identified as part of a sample, they were considered “candidates” 
but not yet the final sites for the case studies. To become final, the sites had to satisfy a 
screening procedure. On October 8, 2000, the program manager for CSRD at the 
Department of Education sent a letter to each school principal, district superintendent, and 
state school official for the 108 schools, informing them that their schools had been 
selected as part of the sample, and that they might be receiving a phone call from the 
evaluation team. Following the receipt of OMB clearance on October 18, 2000, the 
evaluation team began screening the candidates. 


Screening for Confirming School Characteristics. Using a screening protocol (see 
Attachment A-1), the evaluation team telephoned the district contact person for the first set 
of 18 schools in the remaining sample. With the district contact’s approval, the screener 
called the school contact to verify the features listed in the SEDL database, including the 
name of the model adopted by the school. 


Exhibit A-5 documents the shift between the first set of 18 schools screened and the 
final set of schools selected for site visits. The screening process identified one district 
where both schools that were initially selected were no longer using the model identified in 
the SEDL database, but appropriate replacements for these schools were identified within 
the district. Additional issues affecting the final site selection are discussed below. The 
final confirmation of the 18 sites to be included in the study occurred in late November 
2000. 


Issues Affecting Final Site Selection. Two issues affected the final site selection of 
the schools for the evaluation, the inclusion of locally developed models and the 
geographic dispersion of the sites. 


Initially the team intended to include two schools using locally developed models in 
the final sample. According to the database received by SEDL in July 2000, only about 8 
percent of the schools receiving CSRD funding were using locally developed models, and 
the evaluation team believed that the inclusion of two schools using such models would be 
representative of the overall sample. Upon further direction from the Department of 
Education however, two additional locally developed models were added to the sample 
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Exhibit A-5 


RESULTS OF SCREENING PROCESS THROUGH SELECTION OF SITES 


First 18 Schools Contacted 


School and 
Model 


School A 
Locally developed 


School B 
Lightspan 
School C* 
Co-Nect 


School D 
Success for All 


School E 
Accelerated Schools 
School F 

Lightspan 

School G* 

Locally developed 


School H 
Coal. of Essential 
Schools 


School I 

Core Knowledge 
School J* 
Success for All 


Type and 
Region 


Secondary 
Midwest 
Elementary 
Midwest 


Elementary 
South 


ED asked for greater 
geographic dispersion 


Elementary 
South 
Elementary 
Midwest 
Elementary 
South 


Elementary 
South 


Secondary 


West 


Middle 
South 


Elementary 
Northeast 


Reason 
Eliminated 


Planning to change 
state assessments 


Lost district match 


Lost District match 


School no longer using 


Lightspan 


insufficient 


ED asked for greater 
geographic dispersion 


*Indicates that school was selected for case study. 


Student data appeared |School H* 


Second School Contacted 


School and Type and Reason 


Model Region 
School no longer uses 


School A Combined 

Accelerated Schools Midwest Accelerated Schools 
School B* Secondary 

High Schools that Work South 


Elementary | School no longer using 
Success for All West Success for All 


School E* Combined 
Coal. of Essential Schools Midwest 
School F* Secondary 
Coal. of Essential Schools South 


Elementary 


Eliminated 


Success for All 


South 


School I* Elementary 
Lightspan Midwest 


Third School Contacted 


School and 
Model 


School A* 
Accelerated Schools 


School D* 
Roots and Wings 


Type and 
Region 
Elementary 
Midwest 


Elementary 
West 


(Continued on next page) 
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Exhibit A-5 (Continued) 


School and 
Model 
School K* 
Locally developed 


School L 
High Schools that 
Work 


School M 
Direct Instruction 


School N 
Success for All 


School O 
Co-NECT 
School P 

Core Knowledge 


School Q 
Direct Instruction 


School R 
Locally developed 


First 18 Schools Contacted 


Combined 
Northeast 
Combined 


South 


Elementary 
South 


Reason 
Eliminated 


ED asked for greater 
geographic dispersion 


ED asked for greater 
geographic dispersion 


Second School Contacted 


School and Type and Reason 
Model Region Eliminated 


School L* 
Locally Developed 


Elementary 


West 


School M* Elementary 
Lightspan Midwest 


Elementary |Student data appeared {School N* Elementary 
West insufficient Roots and Wings' South 
Elementary |School no longer using |School O* Elementary 
South Co-Nect Co-Nect* South 
Elementary | No match for the model |School P* Secondary 
Midwest High Schools that Work South 


Elementary 
South 
Elementary 
South 


ED asked for greater 
geographic dispersion 


ED asked for greater 
geographic dispersion 


School Q* Elementary 
Accelerated Schools Midwest 


School R* Secondary 
Locally Developed Midwest 


Third School Contacted 


Type and 
Region 


‘School identified the model to screeners as Roots and Wings; in fact school is using Success for All 
*School originally intended to adopt Co-NECT, but switched to Success-in-the-Making 
*Indicates that school was selected for case study. 


because: 1) Department of Education records indicated that 15 percent of the models used 
by CSRD grantees were locally developed; and 2) the Department expressed interest in 
learning more about implementation of these models. For this reason, locally developed 
models now represent 22 percent of the models being used by schools in the final sample. 


Following the first round of screening calls, 12 (or 67 percent) of the 18 schools in the 
sample were located in the southern United States. The sample was presented to the 
Department of Education in early November 2000. ED was concerned with the over- 
representation of schools from this region, and asked the evaluation team to identify 
schools in other regions. The team conducted additional screening calls and was able to 
reduce to eight (or 44 percent) the number schools in South. 


Due to other sample constraints (particularly the need for two schools in the same 
district, and the need for two or more schools to be using the same model), the final 
selection does slightly over-represent schools in the Midwest and underrepresent schools in 
the West and Northeast (see Exhibit A-6). Also over-represented in the final selection are 
high schools and schools in large center cities. Middle schools, schools in rural areas, and 
schools receiving greater CSRD funding are underrepresented. Exhibit A-7 provides more 
detailed information about the final sample. 


Finally, the schools selected for site visits are collectively implementing six of the top 
ten models as ranked in Exhibit A-3: Success for All (1), Accelerated Schools (2), 
Lightspan (3), Coalition of Essential Schools (6), High Schools that Work (7), and Co- 
NECT (8). Two schools are implementing models outside of the top ten: Roots and Wings 
(17) and Success-in-the-Making (also referred to as Computer Curriculum Corporation 
(30). Four schools are implementing locally developed models. Only four of the top ten 
models are not in use at the schools selected for case study: Direct Instruction (4), 
America’s Choice (5), Core Knowledge (9), and HOSTS (10). 


Conducting the Site Visits 


Setting the Agenda. The evaluation design called for two visits to each school per 
year, for each of two years. Initially, the team believed the first visit would occur in the fall 
of 2000. Due to delays in obtaining OMB clearance and issues with the sample selection 
discussed previously, scheduling of site visits was not able to begin until late November. 
Despite the late start, and the difficulty scheduling visit to schools around winter break, 
visits to eight of the districts (16 schools) were completed by the end of January. The visit 
to the ninth district was postponed due to testing at the site, and was conducted in mid- 
February. 
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Exhibit A-6 


AGGREGATE PROFILES OF SCHOOLS SELECTED FOR SITE VISITS, 
COMPARED TO SCHOOLS IN STRATIFIED SAMPLES, 
THE 50% CUTOFF PORTION OF THE UNIVERSE, AND THE TOTAL UNIVERSE)” 


No. of |__School Type’ | Cohort | award | Pov. | Title 1 Area Type’ Region 
Sample Level Schls.| E|M|S |C} 1 | 2 Size Rate | School | SW | 1 


2 7 |MW/NE| S |W 

Final Sample Selected for Visits} 18 |67) O | 22/11) 28) 72 | $63,892] 75 78 61 | 44} 0 O | 33 | 11 | 44) 11 

Sample Selected for Screening 108 | 52] 24)17] 8 | 35] 65] $71,046] 73 86 76 |45} 0 }23)16;} 5 }10] 2 | 19 |} 21 | 47/13 
0 5 


Sample strat. so more than one 569 | 66] 11/12] 12] 39] 61 | $69,785] 75 87 68 | 36 22 21 | 45) 12 
schools is in each district 


Subtotal Retained (50%) + local 945 |63]11]12 | 14] 31) 69) $70,405] 72 93 66 | 23] 2 | 23] 22/11]14] 5 23 16 | 44/16 
models 
Total 1688 | 63} 13] 11]13]| 28] 72 | $69,471] 70 86 65 | 25] 2 |22|)19|/11]13] 6 23 15 |44/17 


Note: ‘Excludes AVID (n=26). Models associated with DePaul University (n=39), Communities for Learning (n=29), Middle Start (n=21) and 
locally developed models (n=106). 
*All entries are percentages, unless otherwise noted. 
°E=Elementary; M=Middle, S=Secondary; and C=Combined grade levels. 
“1=Large center city; 2=Large town; 3=Mid-size center city; 4=Rural; 5=Small town; 6=Fringe of large city; 7=Fringe of small city. 


Source: SEDL Database, as of July 2000. 


£00z ‘uonvsodi0o> SOWSOD 


vI-V 


Exhibit A-7 
FINAL SELECTION OF SCHOOLS FOR INCLUSION IN THE CASE STUDIES 


School Award Pov. | Title 
Name CSRD Model Type Cohort | Amount($) | Rate I? Title I Type Locale 


Reg 
School A Midwest Elementary 1 73 Large Central City 
School B Secondary Prof [Small Town 

School C Elementary Urban Fringe of Large City 
School D Elementary Large Central City 
School E Secondary Large Central City 
School F Secondary zie Urban Fringe of Large City 
School G Elementary Urban Fringe of Large City 
School H Elementary Large Central City 
School Elementary 

School J Elementary | 

School K Combined Grades 

School L Elementary 2 

School M Elementary 
School N Elementary 
School O Elementary 
School P Elementary 
School Q Elementary 


School R_ [Locally Developed Midwest Secondary 


NIN] olo}]w]o aN 
PIN ]aolN}o]e a) 


50,000 


Mid-size Central City 
0 Mid-size Central City 
Mid-size Central City 
Large Central City 
Large Central City 
Large Central City 
Large Central City 
ES ee Small Town 

Mid-size Central City 

no 


Mid-size Central City 


“NI 


[oe] 


C1] O/O];]O]o]o 
WIN] @}]M] OO] O01] P 


iN 
oO 


Source: SEDL Database, as of July 2000. 


Teams of three evaluators were identified for each site. The lead team member 
(generally the most experienced team member), contacted the district representative to 
establish an agenda for the visit. During the first site visit, teams were asked to focus on 
the first four components of CSRD (the model, comprehensiveness, professional 
development, and measurable goals and benchmarks). The lead team member and the 
district representative together identified appropriate personnel for these interviews. At the 
district level, it frequently included the superintendent or deputy superintendent, a 
professional development or curriculum coordinator, a Title I and CSRD coordinator, and 
an evaluator. The district contact was also informed of the team’s need to gather student 
achievement data, and requested the opportunity to speak with the data analyst who 
compiled those numbers. 


Depending on the preference of the district representative, the team also directly 
contacted the schools’ principals to establish an agenda while on site (in some instances, 
the district representative preferred to make that contact). At the school level, interviews 
were scheduled with the principal, assistant principal, any curriculum specialist on staff, a 
school-based model coordinator (where appropriate), and when possible a representative 
from the model developer. The team also requested permission to observe several math 
and reading classes and to interview teachers. 


The teams followed a similar procedure in scheduling the second site visits, which 
occurred in April and May. During the second visit, the teams were asked to focus on the 
remaining components (staff support, community and parent involvement, external 
technical assistance, evaluation of coordination of resources). In most instances, the team 
met again with a district level evaluator, the CSRD coordinator, and a district level budget 
person, along with anyone who could answer questions not addressed during the first site 
visit. At the school level, team again asked to observe classrooms, and interview teachers, 
the parent liaison, a small group of parents, and the principal. 


The site visits were scheduled over a three-day period: one day at the district office 
and one day at each school. 


On-site Protocol and Activities. Members of the site visit team attended a training 
seminar conducted by COSMOS Corporation in November 2000. During the training, the 
teams reviewed the site visit protocol (see Attachment A-2), and discussed what the 
evaluation hoped to gain by the case studies. One important discussion emerged around the 
possible differences between the models that schools adopted and the presence of 
comprehensive reform. Field teams were reminded that the model was only one of the nine 
components of CSRD, and that their task was to collect evidence of comprehensive 
schoolwide reform around all nine components. 


The site visit protocol addressed each of the nine components of CSRD, as well as 
district and state practices and policies related to schoolwide reform at the school, student 
performance, and other contextual conditions that may have impacted the reform efforts at 
the school. The protocol was designed to be used as a guide for the field teams, and not as 
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a survey instrument. In addition, field teams were instructed to seek a variety of 
converging evidence to determine why and how an event occurred, and not simply rely on 
answers to open ended questions. The focus of the protocol study is on tracking and 
document actual behaviors and events, not just as they might be reported by respondents to 
a survey. Other evidence was frequently available in school site plans, CSRD applications, 
meeting agendas, newsletters, and other similar documents. 


Teams were also instructed to begin identifying possible rival explanations for further 
investigation during subsequent site visits. An important objective in the case studies was 
to try to define and test rival explanations for the observed outcome to the extent possible. 
The more hypotheses that were tested and refuted, the greater the confidence that could be 
place in any observed relationship between the CSRD effort and improved student 
achievement. 


While on site, the team conducted interviews (generally 45 minutes to one hour in 
length) with those persons identified during the scheduling process. The team also visited 
an average of six classrooms at each school. The teams requested the opportunity to 
observe classrooms in varying stages of model implementation (i.e., a classroom where the 
model was fully in use, and one where the model was not in use). Team members utilized a 
classroom observation form to record observations during the 45 minute period, and 
attempted to speak to the teachers whose classrooms they observed at some point during 
the visit. 


Finally, either while on site, or during a phone interview, the team requested 
information from the model developers about the fidelity of each site to the original model. 
In nearly all cases, the model developer was able to provide the team with either a written 
or verbal assessment of each schools fidelity. 


Preparing Databases and Summary Reports. Following the site visits, the teams 
prepared databases of evidence, following the outline of the CSRD protocol and citing 
interviews and documents throughout. These databases were then used to develop 
summary reports about each school. The reports were sent to each school and district 
contact to review for the accuracy of the data. The reports were then used to guide the 
development of this first annual report. 
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Attachment A-1 


Instructions for Screening Schools 
for Case Study Selection 


INSTRUCTION FOR SCREENING SCHOOLS 
FOR CASE STUDY SELECTION 


The purposes of the screening procedure are: 
1. To confirm basic information about the school; 


2. To determine the availability of student achievement data about the 
school’s students; and 


3. To identify any other (administrative) conditions that might create 
problems for conducting a case study of the school’s CSRD-related 
activities. 


Note that the screening procedure should avoid inquiry about other educational 
conditions at the school that might influence the selection process. The final set of schools 
is intended to be a representative sample of all CSRD schools. Thus, for instance, finding 
that the school was having difficulty in implementing the CSRD model is intended to be an 
outcome of the final case study—not a reason for not selecting the school in the first place. 


Similarly, the screening procedure is not intended to be an opportunity for schools or 
districts to decline to participate in the case studies. All schools are to be informed that 
they were chosen randomly from the entire set of CSRD schools (with the exception of 
being paired within the same districts), and that their participation in the case studies is 
required as part of their CSRD award [NEED to confirm this, with reference to CSRD 
program in DoED]. 


All screening information is to be summarized in the Site Screening Form 
accompanying this instruction. 


Making Initial Contact 


Because there will be two schools chosen in a given district, the case study team 
should initially seek contacts with both a person in the district and a person in each of the 
schools who is knowledgeable about (if not intimately familiar with) the CSRD program 
(i.e., three persons for every pair of schools). 


For each school, the CSRD database lists at least one such contact person (either in 
the school or in its district), who should be considered the initial person to whom a 
telephone call should be made. This person should then be asked to identify the other 
contact persons, if they were not already listed in the database. (These persons need not be 
the ones who will give final permission for the schools’ participation in the case studies.) 
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1. Confirming Existing Information about the School 


The screening should begin by confirming several ascriptive characteristics of the 
school, as such characteristics appear in the CSRD database (the data will be provided in 
tabular form to the case study teams before the screening process starts): 


@ The school’s status as an elementary, middle, high, or “mixed” 
grade school; 


@® Its CSRD award size and whether it was a Cohort 1 or Cohort 2 
awardee; 


@ Its status as a Title I school and as part of Title I schoolwide; 
@ Whether it “self-reported” itself as a low-performing school; 


@ Whether it is located in the “area type” defined by the database; 
and 


@ Whether it adopted the CSRD model listed in the database. 


Where discrepancies are uncovered, the school should be reminded that the CSRD database 
was derived from the school’s own response to a SEDL questionnaire [NEED dates for 
these questionnaires], just to make sure that there is an adequate explanation for the 
discrepancy(ies) (e.g., the reality changed after the school submitted the original 
questionnaire). Further, in carrying out this part of the screening inquiry, the existence of 
discrepancies should not be conveyed as a reason that the school might be dropped from 
the case study sample. 


The contact person should be asked to forward (e.g., fax) to the team any written 
information about the school, covering these and related characteristics, including a copy of 
its CSRD application (the application may be available from the school or the district). 


2. Other Administrative Conditions 


The contact person(s) should be asked about any disruptive administrative conditions 
that might make a case study infeasible. Examples might be: 1) sudden or unexpected 
turnover in key leadership positions (e.g., principal); 2) a recent and traumatic incident, 
such as an act of youth violence, that might have put the school into the public and media 
eye; 3) a recent district or court takeover of the school, putting it into a limbo status. 
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SITE SCREENING FORM (attach notes, where needed) 


1. Contact History: 
Date(s) of Screening Contacts: 


Evaluation Team Member Making Contact: 
Person(s) Contacted (name, title, and phone number): 


2. Name and Address of School: 


3. Information to be Confirmed from SEDL Database: 


a. School Status: (filled out from database) 
b. CSRD Award Size: (filled out from database) 
c. Receipt of Title I Funds: (filled out from database) 
d. Low-performing school: (filled out from database) 
e. Area type: (filled out from database) 
f. CSRD Model adopted: (filled out from database) 
g. Name of school district: (filled out from database) 


circle one; if no, write note 
of explanation 


Confirmed: 
Confirmed: 
Confirmed: 
Confirmed: 
Confirmed: 
Confirmed: 
Confirmed: 


yes 
yes 
yes 
yes 
yes 
yes 
yes 


no (note) 
no (note) 
no (note) 
no (note) 
no (note) 
no (note) 
no (note) 


4. Possible Disruptive Administrative Conditions Making Case Study Infeasible (discuss with note) 


Attachment A-2 


Case Study Protocol for 
CSRD Field-Focused Studies 


OMB Clearance #1875-0188 
Expiration Date: 09/30/2002 


CASE STUDY PROTOCOL FOR 
CSRD FIELD-FOCUSED STUDY 


I. Introduction 


This protocol is to be used by field teams in carrying out the field-focused study 
portion of the CSRD evaluation. The study is to collect detailed and intensive data from 
specific CSRD schools, taking the form of case studies of each school. Each case study 
will contain both qualitative and quantitative data. 


The field-focused study addresses four questions: 


1. What has been the course of implementing the CSRD mandate 
in CSRD schools, and with what likely effect on student 
achievement? 


2. Under what conditions are schools successful in implementing 
schoolwide reform? 


3. What external activities and changes are important to CSRD 
schools—e.g., 
-district and state support strategies; and 
-external technical assistance? 


4. What do case studies tell policymakers about promising models, 
programs, and technical assistance/support strategies? 


The protocol serves as a standardized agenda for collecting process and outcome data. 
The intended focus is to define actual events and behaviors, not perceptions and attitudes 
alone. Thus, field teams must seek a variety of converging evidence to determine whether, 
why, and how an event occurred. In this sense, the methodology differs from those relying 
merely on a series of open-ended interviews. 


Overview of the Protocol 


CSRD Theoretical Framework As a Starting Point. The organization of the protocol 
follows the theoretical framework (“logic model”) developed in the evaluation design for the 
field-focused studies (COSMOS/TMG, January 2000). Exhibit 1 depicts the framework and 
identifies the key components (items in the “boxes”) and their presumed causal 
relationships (“arrows”), reflecting CSRD’s definition of schoolwide reform. Complete 
reform is presumed to produce improved student performance. 
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Exhibit 1 


OMB Clearance #1875-0188 
Expiration Date: 09/30/2002 


THEORETICAL FRAMEWORK FOR A CSRD SCHOOL 
(COSMOS-TMG EVALUATION) 


(A) The research -based method and the 
main outcomes of interest: 


1.* Research- 
based 
method 


4. Student 
performance 


(C) Presumed causal flow among the components: 


2. Comprehensive 
and aligned 
design 


5. Support by 
faculty, etc. 


School Organization: 
4a. Measurable goal 
and benchmarks 


3. Professional 
development 


Instruction 
and learning 


6. Parental and 
community 
involvement 


* = The numbered boxes refer to the nine CSRD components. 


(B) School actions affected by research -based method 


(vary by model): 


5. Support by 
faculty, etc. 


School Organization: 
4a. Measurable goals 
and benchmarks 


3. Professional 
development 


6. Parental and 
community 
involvement 


2. Comprehensive 
and aligned 
design 


1:* 
Research- 
based 
method 


4b. Student 
performance 


Instruction 
and learning 


(D) Addition of final three components: 


9. Coordination 
of funding 
resources 


5. Support by 
faculty, etc. 


2. Comprehensive 
and aligned 
design 


¢ 


3. Professional 
development 


a 6. Parental and 7 
community 


involvement 


7. External support 
and assistance 
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1.* 
Research 
based 
method 


4b. Student 
performance 


Instruction 
and learning 


8. Evaluation 


OMB Clearance #1875-0188 
Expiration Date: 09/30/2002 


Additional Topics to be Investigated. Not explicitly specified by the CSRD program 
and therefore missing from this initial framework are external conditions that nevertheless 
need to be investigated. These external conditions primarily refer to district (LEA) actions, as 
well as possible state department of education (SEA) actions. To obtain these data, field 
teams will therefore conduct site visits to the CSRD school’s host LEA, and possibly make 
telephonic contact and obtain documents from the SEA. 


Time Perspective. Critical to the entire data collection effort is to track events 
chronologically, to assist in making attributions. Especially important is to identify whether an 
event occurred during: 


1) The academic year or years (June to June) prior to the 
onset of the CSRD award; 

2), 3), 4) The 1st, 2nd, or 3rd CSRD years; or 

5) The year after the final CSRD year. 


Field teams should expect some CSRD-related actions (e.g., the adoption of a CSRD model) 
to have started prior to the CSRD award, as schools already may have been moving toward 
some aspects of reform before CSRD. Similarly, teams should watch for signs of 
sustainability in the year after the final CSRD year. 


Rival Explanations. Also critical is for teams to identify and investigate rival 
explanations. “Rivals” are events not related to CSRD that may have produced (or co- 
produced) CSRD-related outcomes. Suspicions about such rivals are likely to arise during 
the course of the fieldwork, although information collected from the developer prior to the site 
visit (See “Preparation for Site Visits,” below) also may be insightful. Of greatest interest are 
any rivals, or alternative explanations, for changes (or lack of change) in student 
performance. Teams should aggressively seek data in support of rivals. Under this 
condition, the more rivals that have been identified but then rejected because of lack of 
support, the more that effects can be attributed to CSRD actions. 


Data Collection Procedures 


Repeated Site Visits to the Same Schools. Field teams are to arrange for site visits 
to the target schools. Each school will be visited twice during an academic year, for two 
consecutive years. During the first year, the entire protocol should have been covered and a 
first-year case report prepared. During the second year, emphasis should be on updating 
the first-year report. Therefore, the same topics are to be covered, but the line of inquiry 
should focus on the incremental changes that have occurred during the preceding time 
period. (The protocol topics have not been worded to differentiate the first- and second-year 
orientations, so field teams must themselves customize the words.) 
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During the site visits, interviews with specific individuals need to be arranged (school 
staff, consultant or specialist external to the school, and parent leader or representative). 
Attachment A of the protocol contains a crosswalk of the functions of these individuals, as 
they pertain to the topics of inquiry. For larger schools, these functions are expected to be 
distributed among a larger group of people, whereas for smaller schools, they are likely to be 
embodied in a small group. 


In addition, the crosswalk also indicates where classroom observations should be 
made. The classroom topics will be covered by selecting three types of classrooms: 1) one 
in which the implementation of CSRD schoolwide reform has been the most advanced; 2) 
one that is transitioning from pre-reform practices and norms; and 3) one in which 
implementation has not yet begun. Attachment B of the case study protocol contains the 
classroom observation form to be used. 


Preparation for Site Visits. Prior to any site visit, field teams should collect and 
analyze data about the target school. This information will have come from several sources, 
including: the SEDL database (which contains a brief profile of every school; the data 
collected during a site screening process that led to the selection of the school in the first 
place; and information about the specific CSRD model that was adopted, based on (interview 
as well as documentary) data collected from the model developer. In making site visit 
arrangements, the team also should have requested documentary materials from the school, 
to be sent to the team prior to the site visit. 


Field teams are to use all of this information to detail and customize the original CSRD 
framework. For instance, depending upon the CSRD model that was adopted, different 
relationships are anticipated among the other eight CSRD components (The illustrative 
exhibits in Attachment C depict the beginning of this customization process.) The fully 
customized framework should then provide a concrete idea of the line of inquiry to be 
pursued, before actually starting a site visit. 


Assembling of Evidence and Preliminary Case Study Reports Immediately 
Following Site Visits. Field teams are urged to begin the formal analysis and report-writing 
process as soon as a Site visit has ended, though additional data may still have to be 
collected. Assembling data and drafting narratives proceeds more efficiently and with much 
higher quality if this time sequence is followed. Teams should reserve the day or two after 
the site visit for this activity, avoiding other commitments. 


Outline of Case Study Report. The topics of inquiry are organized according to three 
levels, referenced by headings (e.g., Component 1, etc.), letters (e.g., A, B, C, etc.), and 
numbers (e.g., 1, 2, 3, etc.). The case study report should follow this same heading 
structure. Under each topic and subtopic is expected to be some narrative material (with 
citations to sources of evidence). Appended to the report should be: 1) a chronology (about 
25-30 of the most important events); 2) a list of references reflecting the citations in the 
report, and 3) a list of field documents stored in the team’s own files. 
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ll. Topics of Inquiry 


As a general data collection strategy, priority should be given to: the main intervention, 
student performance data, and the apparent connection between the two. Lower priority 
should be given to other topics (unless directly related to CSRD), such as: a school’s history, 
neighborhood conditions, or other contextual events. (Remember that the final case report is 
about CSRD-related schoolwide reform and is not intended to be a case study of the school.) 


The topics below not only cover the “agenda” to be followed by the field team but also 
give explicit probes and examples regarding the type of evidence that is being sought. Asa 
result, the protocol should provide guidance on how to know what to look or listen for, and 
how to recognize relevant evidence’ when it is encountered. 


Component 1: The CSRD Model 

The school should have adopted and implemented a model practice (“the CSRD 
model”) that is based on reliable research and proven methods. Ideally, the model already 
will have been replicated successfully in schools with diverse practices. 


A. Define the educational practice or model adopted by the school, as a result of its 
participation in CSRD. Identify the model’s name, the version (if any) being adopted, and the 
“locus of intervention” (by operationalizing the boxes in Exhibit 1) that the school intended or 
needed the model to cover. (A key portion of this information should come from data 
collected from the model developer; however, be alert that the school may have deliberately 
customized or adapted the model.) 


B. Establish when the school began to consider adoption, and whether the course of 
events’ was limited to CSRD-instigated initiatives or whether other (“rival”) conditions® also 
accounted for adoption. 


C. Ascertain whether only one model was adopted (the “CSRD” model), or whether 
additional practices or models were deliberately coupled‘ with the CSRD model as part of the 
school’s overall reform strategy. (Companion models might take any form—including being 
another of the models on the original CSRD list.) Where companion models are found, 
describe the relationship and rationale for the combination of the CSRD model and the 
companion model(s), with regard to educational objectives and locus of intervention. 


D. Itemize the specific school operations involved in implementing the model. For 
example, for “process” models, show how class scheduling, school management, or other 
administrative or educational processes have changed as a result of the implementation of 
the model. Alternatively, for “prescriptive” models, identify the specific departments or 
classrooms (with estimated enrollments) where implementation of the model has been 
occurring. 


* Note areas where evidence is weak in preparation for next visit. 

* This is a timeline of events. Actual support should appear under Component 5. 

3 For example, what were they using prior to this model. 

* Additional models should be in a related subject area. Models covering different subject areas belong 
under Component 2. 
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B. Describe the quality and breadth of the school’s plan’ for schoolwide reform 
[obtaining the plan would be helpful]: 


1. How does it define the main objectives for reform? 

2. Was any empirical evidence (e.g., needs assessment) collected? 
3. What strategic priorities were laid out? 

4. Which of the eight components appear in the plan? 


C. Define any planning, coordinating, or other management processes whereby the 
eight components are being linked conceptually or administratively, both to each other and to 
the same vision, if any, for schoolwide reform. 


D. Provide actual examples of coordination or linkage among the components, such as: 


! Initiating professional development activities (Component 3) 
that are tied to methods for achieving measurable goals 
and benchmarks (Component 4); 

! Promoting school support activities (Component 5) for the 
CSRD model being implemented (Component 1); 

! Promoting parent or community involvement (Component 6) 
in relation to the CSRD model being implemented 
(Component 1); or 

! Defining relevant evaluation data and strategies 
(Component 8) to provide feedback on measurable goals 
and benchmarks (Component 4). 


E. What is the evidence, if any, about the alignment of the components, e.g., 


1. Evidence that standards, the research-based model, 
instruction, and assessment have been judged (by others) to be in 
alignment with one another, such as: 
-textbook and instructional materials reflecting the 
standards of the state or district; 
-state and local assessments have been reviewed for align- 
ment with academic standards; and 
-evidence of teacher familiarity with the standards and 
general agreement over their content. 


F. To what extent is the comprehensive plan and design for the school understood and 
embraced by faculty and parents and used to guide day-to-day operation? [The more that 
examples can be found in response to topics B-F, the more aligned the reform effort is 
considered to be.| 


> Actual professional development activities should be described in Component 3. 

° Write this section of the report last. Do not repeat details from other sections of the report. 

” Comment on the breadth and depth of the design, and consistency with state and district policies, 
without describing each component. Decide whether the design involved the whole school, or was 
limited to one subject area (like SFA reading). 
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° The concept of design goes beyond the school improvement plan; describe more than just the plan. 
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Component 3: Professional Development” 
Relevant professional development includes not only teaching staff but also 
administrative staff. 


A. Document the amount of time, per teacher or staff person, devoted during the past 
academic year to professional development,”° and estimate the cost of such professional 
development. Within these overall totals, what proportion™* was explicitly devoted to CSRD- 
instigated schoolwide reform (e.g., by definition, professional development that started prior 
to CSRD and that has continued since then may not be directly related to CSRD)? 


B. To what extent can it be said that schoolwide reform has led to a changed 
structuring or implementation of professional development activities as a whole? 


C. Enumerate the professional development initiatives explicitly devoted to CSRD- 
instigated reform (not the individual workshops, class coaching, ? or professional 
development sessions, if avoidable), indicating: a) their length and frequency; b) the number 
of teachers or administrators involved; and c) the substantive connection to schoolwide 
reform (if possible, identify the component or components covered). [Obtaining copies of 
workshop agendas, participant lists, and outlines of substantive topics covered would be 
helpful.| For each of the professional development initiatives deemed most important to 
schoolwide reform, cover the following questions. 


1. What were the qualifications of the professional development 
organizers and trainers—e.g., were they representatives of 
the CSRD model's developers? 

2. What evidence is there about the participants’ attendance, 
reactions, learnings, or action plans? 

3. How explicit was the initiative in addressing potential conflicts 
between the “old” and “new” ways of doing business, and 
how to discard or replace the old ways? 

4. Describe the follow-up professional development activities, if 
any, that ensued, especially after school staff attempted to 
implement the CSRD model. 


D. Comment on the adequacy of the reform-oriented professional development 
initiatives, regarding the number of participants, amount of professional development time 
and opportunity per participant, and relevance to actual reform challenges and needs at the 
school. 


° This section should describe professional development from the point of view of the recipient. Comments 
about external technical assistance belong in Component 7. 

*° Include internal techniques like lead teachers and modeling in classrooms 

‘7 Comment on whether CSRD is on top of, or part of, routine professional development activities and 
funding. 

2 Comment on who pays the salary for the internal coach. 
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E. What is the evidence that the reform-related initiatives have led to: 


1. Deepening of teachers’ knowledge of academic content; 
2. Encouragement of teachers from the same grade levels 
or departments (Subjects) to work in teams; and 

3. Meeting the substantive needs of the school staff. 


Component 4: Measurable Goals and Benchmarks 

Reform is believed to require schools to set measurable benchmarks, both for 
implementation progress (process) and for improved student performance (outcomes). Such 
benchmarks should be embedded within an accountability system that provides incentives 
for success and remedies for failure. 


A. Provide the timeline and benchmarks, if any, that reflect the implementation of the 
reform (e.g., scale-up as reflected by the number of classrooms, grades, teachers, or school 
subjects affected by standards-based or model-related criteria; the dates for progressing 
from the planning of reform to its initial implementation and then to more refined and 
operational implementation). How clear and detailed is this timeline, and is it being tracked 
regularly? [Use of the school’s own exhibits or documents would be helpful.] Within this 
framework, comment on whether implementation of reform appears to be in the planning, 
piloting, early implementation, or late implementation stages. 


B. List the benchmarks,’® if any, that the school has adopted to represent measurable 
reform goals and objectives over time (e.g., annual goals or five-year goals matching a 
strategic planning period). [These can include both process and student outcome goals— 
e.g., teacher-related goals or student-related goals such as enrollment in particular courses, 
attendance and dropout rates, and achievement.] 


C. Comment on the extent to which the goals and objectives are actually measurable, 
and determine whether relevant and compatible measurement systems are in place to 
monitor this progress (e.g., regular schedule of classroom visitations to determine changes in 
instruction, repeated surveys of parents and teachers regarding their observed changes, and 
review of student progress). [/f adequate measurement systems are in place, you should be 
able to collect actual baseline data.| 


D. For the student-related benchmarks, if any, comment on the extent to which they 
are mainly cast in terms of the average student, compared to advances to be made by all 
students. 


E. Define the specific incentives for success in achieving goals and benchmarks or the 
specific remedies when failing to do so, if any. 


*S Do not comment on the evaluation efforts, or result of any data analysis in this section; that belongs 
under Component 8. 
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Component 5: Support™ within the School 

The breadth of schoolwide reform demands that teachers, administrators, and others in 
the school all support the reform (some of the CSRD models also require evidence of specific 
forms or levels of support). 


A. What have been the faculty meetings or votes, if any, on the reform activities, and 
when did they take place and with what result (e.g., support or opposition)? 


B. How instrumental has the principal been in leading or participating in the reform effort 
(give examples of specific initiatives or decisions made by the principal)? Comment on 
whether the principal should be regarded as the leader of the reform effort. 


C. What changes in school schedules, operations, or procedures have been made, if 
any, to accommodate the reform? 


D. What evidence is there that real and cherished resources have been called upon to 
support reform activities (e.g., Title | dollars have been redirected or staffing patterns have 
been altered)? 


E. Give examples of the “language” of reform at the school—e.g., the extent to which 
administrators and teachers speak about the overall reform as a permanent change, not 
passing phase; the depictment of reform in school announcements, newsletters, and other 
outreach documents. 


F. Comment on the extent to which the responses to the preceding five questions reflect 
genuine support for genuine reform. 


Component 6: Parent and Community Involvement 

Parent and community involvement can lead to support for reform in at least two 
important ways: in instructional roles (direct involvement in the education of the child) and in 
governance roles (direct involvement in school decisionmaking). 


A. What is the evidence that, over time, the role and number of parents and community 
representatives involved in the education of the child (e.g., tutoring, providing classroom 
assistance, working with the child in doing homework) has changed? Are formal records kept 
by the school? [Examples of such records would be helpful] 


B. To what extent have any of the above activities been specifically geared to the reform 
effort (e.g., parents have been directed to work on the educational subjects that are of high 
priority in the reform activities)? 


* Include all information about the extent of support in the discussion of this Component. 
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C. How have the roles and influence of parents and community representatives in 
positions of school governance” (e.g., school board and PTA) changed, if at all, in relation 
to CSRD-initiated reform? 


D. In addition to any of the preceding topics, how has volunteer time by parents or 
community members changed in relation to the reform? 


E. Outside of the school system, to what extent have other entities voiced support for 
or opposition to the reform (e.g., announcements and articles in local newspapers, 
contributions by the business roundtable, or support or policies by the local aldermanic office 
or other neighborhood groups) [related questions about the district and state education 
department are found in Section B of the protocol}? 


Component 7: External Technical Support and Assistance 

The reform components include having the school utilize high-quality technical support 
and assistance from a comprehensive school reform entity (which may be a university 
team). The entity should have had extensive prior experience in implementing school 
reform. 


A. What external’® technical assistance consultant(s) or team(s) have been engaged 
by the school (and with what credentials), specifically to support the CSRD-initiated reform 
design or implementation effort? Describe the arrangement (e.g., the level of funding; the 
length of the engagement; and the existence of a formal agreement”’ between the school 
and the TA team, covering the services and timeline for each activity). 


B. To what extent has the assistance taken the form of regular, on-site assistance, 
and to what extent is there a close relationship between the school and the assistance 
provider(s) [citing or obtaining a schedule of visitations would be helpful|? 


C. Track and enumerate the breadth of the team’s assistance: Has it covered one 
component or more? To what extent has the team been concerned with the 
comprehensiveness of the whole effort (Component 2)? 


D. Describe how well the technical assistance has worked (e.g., provide evidence 
regarding the quality of the assistance, how it has been received by the school, and whether 
there have been examples of any concrete benefits or outcomes). 


Component 8: Evaluation” Strategies 

The school should be using evaluation to monitor and assess the reform activities, with 
the data providing feedback for mid-course corrections as well as assessing the attainment 
of goals. 


*S Parent involvement on school boards and family support teams should be described here. 

’© The information about the developer and other outside consultants belongs here (include source of 
support for costs and whether this is part of the broad costs of professional development described 
under Component 2, or in a addition to those costs). 

’ Describe who decides when external TA is needed. 

*8 Define Evaluation broadly to include all analytic activities and make clear whether formal evaluation 
methods are being followed. 
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A. What kind of evaluation effort is in place [obtaining a copy of the evaluation plan 
and copies of any evaluation reports would be helpful|? Comment on: 


1. Who is doing the evaluation (e.g., a staff person assigned to 
monitor progress or a formal evaluation team); 

2. When the evaluation started, and whether it was conceived 
as part of the initial design of the reform or whether it 
emerged later; and 

3. The technical design of the evaluation (e.g., classrooms or 
activities covered, instruments or measures being used, 
and data collection procedures). 


B. How well does the design of the evaluation map back to the measurable goals and 
benchmarks (Component 3)? 


C. To what extent is the evaluation covering “soft” measures (e.g., attitudes and 
perceptions) in comparison to “hard” measures (e.g., measures of actual performance, such 
as student achievement)? How is the possibility of contamination and bias being 
addressed? 


D. How has the evaluation information been shared (e.g., to school staff only or to 
parents and the community), and how has this information been used (e.g., cite any mid- 
course changes instigated by evaluation findings)? 


E. What is the evidence that evaluation data led to any modifications or decisions 
regarding the implementation of schoolwide reform, or to instruction? 


Component 9: Coordination of Resources 

The CSRD award was not intended to be the sole resource for reform. On the 
contrary, the CSRD award was intended to be used as a stimulant or catalyst for 
coordinating (and potentially reallocating) other school resources, whether involving the core 
budget and other special funds (e.g., Title |, Eisenhower, foundation awards, etc.). 


A. What is the size of the core budget of the school, and can the other major sources 
of funds be identified? 


B. Give examples of how resources have been coordinated and devoted to schoolwide 
reform. Describe how coordination takes place. [Documentation of the coordinating 
process, and how it links with the CSRD-initiated reform, would be helpful. 


C. Whether coordinated or not, what is the evidence that resources have been 
redirected to support reform? 


D. What is the evidence that the school has defined resources to sustain reform after 
the CSRD grant ends—so that the CSRD funding really performs as a stimulant or catalyst 
and does not end up being the entirety of the reform effort? 
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District and State Practices and Policies Related to 
Schoolwide Reform at the School 


A. What district, State, or neighborhood initiatives or conditions might have affected 
the school’s reform efforts [use Exhibit 2 to guide the lines of inquiry]? 


B. What are the signs of support or lack of support for the schoolwide reform at the 
target school (not schoolwide reform as a general principle)? Waivers, access to special 
resources, and peer and collegial consultation all would be examples of such support. 
Distractions to other issues, excessive delays in needed approvals, or demands for 
competing initiatives might all be considered examples of lack of such support. 


Other Contextual Conditions 


A. What has been the school’s experience with regard to enrollment, graduation 
rates, attendance rates, dropout rates, and graduation requirements? [Collect this 
information for multiple years, with the first year being the academic year prior to the CSRD 
award. Collect other data about the school’s performance over time, which would be useful 
to contextualize the reform effort. For instance, this might include a brief discussion of the 
problems leading to reform.] 


B. Have there been any qualitative conditions in the school that might affect 
schoolwide reform and even serve as rival explanations? For instance, a positive condition 
would be that the school had previously been successful in reform-like activities, either as a 
result of private foundation funding or Title | schoolwide funding. Further positive signs 
would be the receipt of awards and recognition for school performance. Negative 
conditions would be that the school had been suffering from poor leadership, high teacher 
turnover, a high degree of student misbehavior, or other difficult conditions that would make 
reform a more difficult accomplishment. 
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Exhibit 2 
EXTERNAL CONDITIONS TO BE INVESTIGATED 


1. Determine whether any district or State policies, if any, appear to have affected the 
target school’s practices in relation to schoolwide reform (only): 


a. What teacher recruitment, certification, or retention policies have been 
put into place that might affect schoolwide reform just prior to or after 
the CSRD award took place? 


. What budgetary changes or other resource policies, including the overall 
level of such resources may be affecting reform? 


. What assessment practices, especially changes in assessment tools 
during the time of the reform effort and tuned to standards-based 
criteria, have been put into place? 


. What policies regarding the adoption or use of curriculum materials, 
reflecting standards-based curriculum frameworks, have been put 
into place recently? 


. Has district leadership changed recently, and with what apparent 
effect (Supportive or not?) on schoolwide reform? 


. What other district-wide or Statewide reform efforts are ongoing 
concurrently, such as those supported by the National Science 
Foundation in mathematics and science education, or the New 
American Schools more generally, that might provide guidance 
to or create demands on the target school? 


2. Define any community or local conditions that might have affected the target 
school’s practices in relation to schoolwide reform (only): 


a. What kind of turnover in neighborhood populations, if any, and 
therefore in the school’s student body, has taken place just prior 
to or after the CSRD award took place? 


b. What economic shifts, if any, leading to markedly different 
support (better or worse) for the school on the part of businesses 
and industry, has occurred at the district or State levels? 


c. What gangs, violence, drug, or other exceptional student problems, 
if any, have occurred that might have diverted attention or re- 
sources away from schoolwide reform? 


Information about these external conditions should be sought by collecting data from 
the district as well as consulting other sources such as local newspapers. 
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Appendix B 


PROMISING REFORM STRATEGIES 


The following strategies were identified by the site visit teams during the course of 
four site visits to each site over a two-year period. The strategies represent practices, 
“programs, and technical assistance/support strategies” that were useful to the grantees in 
their efforts to implement the nine components of CSRD. The strategies were identified 
through interviews with school and district personnel, document reviews, and observations. 
They are not based on any analysis of student achievement data. 


Component 1: Research-Based Method 


School K: Stringent Implementation of the Research-Based Method Believed to 
Result in Improved Reading Scores 


All of the teachers at School K were fully trained in the use of SFA and fully 
implemented the program in their reading groups. Classrooms had all of the SFA materials, 
followed the instructional scope and sequence religiously, and were in all respects an 
exemplary model for SFA. Teachers varied somewhat in their enthusiasm for SFA, but all 
worked very hard to make the reading program work. Teachers indicated they were 
grounded by one goal—to see their children’s reading performance improve. The first year 
of implementation was very difficult, given SFA’s prescriptive approach. Teachers 
persevered, because they felt the more effective use of instructional time would ultimately 
facilitate student learning. This was true for new as well as more experienced teachers. 
Teachers in the school supported one-another and were chiefly responsible for the excellent 
implementation of the model. This level of teacher commitment and effective program 
implementation is exemplary and yet SFA will be replaced in the 2002-03 school year. 
Teachers credit SFA for improvements in student reading in the primary grades, but SFA 
had only marginal effects in the upper grades. Teachers in the upper grades were not able 
to make the changes they thought were needed to improve their student’s performance 
because of SFA’s rigidity. The entire teaching staff withdrew their support of SFA because 
this rigidity and they believed the method developer did not adequately respond to their 
concerns. This teaching faculty exemplifies the importance of “buy-in” and collaboration. 


School J: Research-Based Method Guiding Scope and Sequence of Lessons in All 
Subject Areas 


Many teachers have found the structure and sequence required by the method for 
lesson development a positive addition to their instructional capacity. What is learned from 
developing a specific lesson can be applied to other areas of instruction. Teachers taught 
the value of students self appraisal. This helps the student understand what is expected and 
when they have met all of the requirements of a particular lesson. 
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Several district policies appear to be instrumental in supporting the continuation of 
School J’s reform efforts after CSRD. The first is the reorganization of the central office 
administration, particularly the arrangement of the district’s schools into three geographic 
clusters. Each cluster includes those schools that feed into one of the district’s three high 
schools. An associate superintendent is responsible for each cluster and facilitates 
communication and consistent policies and practices (e.g., a common teacher evaluation 
form) across all schools in the cluster. Teachers and staff within a cluster have 
opportunities to participate in professional development at another school in the cluster, 
exchange information about best practices (e.g., regarding ways to enhance parent 
involvement), and discuss how to coordinate educational programming among schools in 
the cluster. Although in its first year of implementation, the principal notes that the school 
receives greater attention from the district due to the cluster organization and better 
coordination among schools may help to ease student transitions and improve student 
achievement. 


School R: Teacher Teams Provide Students with Cross-Curricular Authentic 
Learning Opportunities 


The 9th grade teams at School R provide an opportunity for teachers to engage in 
flexible block scheduling, interdisciplinary curriculum, and team planning. For example, 
the Freshman teams organized an Environmental Congress for the entire freshman class, 
involving all major subjects: 


— Science — students learned about environmental issues. 
— Math — students work on graphing findings. 


— English — students developed position papers for their proposals 
to the congress, and work on presentation skills. 


— History — students learned about the political process involved. 


The 9-week curriculum culminated in a “Congress” for the entire Freshman class held 
in the school auditorium. Candidates were selected from each team and presented their 
position papers. Members of the audience (other Freshman class members, teachers, 
school and district administrators, and some parents) were asked to comment or debate the 
topic, and then vote on the outcome. 
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Component 2: Comprehensive Design 


School G: School Improvement Planning Process Aligns Reform Initiatives 


In 1999, district staff led workshops to prepare schools for developing school 
improvement plans aligned with budgets and needs assessments. The 2000-01 plan has 
three priorities: improving academic performance, continuing professional development, 
and increasing parental involvement and student attendance. The district seeks to align all 
reform efforts with state standards and district goals and is viewed by one state official as 
one of the most advanced districts in complying with state requirements. The district 
actions were the result of a new state mandate that all school efforts be integrated into 
comprehensive school improvement, reflecting state content standards (e.g., plans must 
reflect state standards). 


Component 3: Professional Development 


School G: Professional Development Activities and Instructional Practice Driven 
by Student Assessment Measure 


Extensive teacher professional development in student assessment was one of the 
most promising strategies observed at School G. Individual and disaggregated student data 
was available for teachers in both reading and mathematics. Teachers at School G received 
extensive job imbedded professional development training on instructional improvement 
driven by student assessment measures. 


Teachers at School G were using data to not only gain an understanding of student 
achievement levels, but also to drive instructional practice. For example, one kindergarten 
teacher, who was participating in an action research project through the local IHE, detailed 
how during the 2001-02 school year she used pre-test data to guide individual and 
classroom reading instruction. Using pre-test data, the teacher was optimistic in citing 
individual student gains during the school year, despite not having received post-test data 
results. Instead of dreading the ensuing test results, this teacher was anxiously awaiting 
confirmation of what she believed would be an affirmation of her expectations. Rather 
than attacking the problem of low level reading ability with a general increase in the 
amount of student reading, teachers at School G gained valuable data-driven professional 
development training opportunities that helped teachers target the skills and subskills 
necessary for teaching effective reading. 


School I: Staff Understanding of Common Planning Period as Professional 
Development Opportunity 


Staff support for reform at School I, and for the school environment in general, 


showed modest increases in 2001—2002, in part because of the creation of collaborative 
planning time. The principal reorganized the schedule to allow teachers the opportunity to 
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meet weekly with others on their grade level to discuss achievement data and instructional 
methods. Teachers commented that this practice, which required no additional funds to 
implement, was beginning to dramatically change the culture in the school. 


The planning time was perceived by teachers as a professional development 
opportunity, and as a show of respect by the school administration for their profession. 
And teachers reported that they were stepping up to this challenge. According to the 
teachers, when they would find time to talk to each other in the past, the conversation 
focused on complaints about students or school and district policies. With the advent of 
scheduled planning time, the teachers report that the meetings are used constructively to 
review data and problem solve weaknesses, or to bounce ideas about instructional 
techniques off their colleagues. 


This shift in practice was aided by the district office, who helped the school staff learn 
to understand and interpret student achievement data. Teachers learned the language and 
skills needed to discuss reading and math scores, and use each other to create strategies. 
The addition of the 4th and 5th grade classes to School I, which brought the school into the 
state assessment system, impressed upon teachers the importance of this shift. 


Component 4: Measurable Benchmarks 


School P: Method-Recommended Curriculum Team Aligning Scope and Sequence 
to Provide Greater Interdisciplinary Project 


The curriculum team, initiated through a HSTW professional development workshop, 
was in the process of overhauling School P’s curriculum framework. The team was 
aligning the sequencing of course topics across disciplines to provide greater opportunity 
for across-discipline projects. The MSW technical assistance report and curriculum audit 
and the MSW standards-based curriculum professional development workshop assisted the 
team in designing a curriculum framework that was both academically enriching and 
systemically consistent for middle and high school students. However, the impending 
school merger interrupted the enthusiasm and proposed intentions of School P’s curriculum 
team, and by Spring 2002 it was unclear whether the efforts would continue. 


School C: External “Critical Friends” Group Provides Objective Assessment of 
Progress Towards Goals and Benchmarks 


The Co-NECT Critical Friends process is a unique form of assessing method 
implementation. Using the Critical Friends process, School C planned an informal (local) 
critical friends assessment process in Spring 2002. The purpose of the progress review was 
to take a “snap shot” of the school’s progress through classroom observations, and to 
provide feedback to faculty and staff that would help to guide goal setting and planning for 
the 2002-03 academic school year. Data collected during the progress review would be 
utilized to enhance Co-NECT implementation and the SIP of School C. The Co-NECT 
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consultant and facilitator, the principal, design team members, a HOSTS community 
volunteer, and a parent comprised were members of the progress review team assigned to 
observe quality teaching, learning, and method implementation. Due to dangerous weather 
conditions, the school canceled the Critical Friends meeting and the site visit team was 
unable to observe the process. 


Component 8: Evaluation 


Schools H and N: Clear Scope and Sequence and Quarterly Assessments Provide 
Frequent Measures of Student Growth 


The district has developed a scope and sequence system for the major academic areas 
derived directly from the state standards. To regularly measure student progress, the 
district conducts system wide assessments every nine weeks in reading, mathematics, 
science, and social studies, based on the scope and sequence documents. 


To implement the assessment, which is used solely for diagnostic purposes, the 
district utilizes a commercially developed system to analyze results and provide feedback 
to teachers and administrators. The system does not contain assessment items that must be 
used; rather, it allows the district and teachers to develop and use items that are tied directly 
to the scope and sequence. 


The system, which is computer based, allows for results to be tabulated and presented 
to teachers and administrators in a variety of ways. Teachers can obtain both individual 
and class results, as well as item analyses to determine if there is a pattern to student 
responses that calls for adjustment in instruction. Teachers across common grades can 
compare results and use the comparison to make adjustments. Generally, teachers have 
access only to results for their students, though, in one school, teachers were allowed 
access to results for the entire school. 


School administrators have access to results for the entire school and can do item 
analyses, compare results at grade levels, and design and provide assistance to teachers and 
students who are having difficulty. 


At the district level, all results are reviewed by district administrators and used for 
professional development, adjustments to scope and sequence, and other educational 


refinements. 


Since the system is Web-based, all users are able to analyze results from various 
viewpoints instantaneously. 


This is the first year the system has been in place so that results in terms of improved 
student performance on the state assessment cannot be determined. 
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Component 5: Staff Support 


School F: Internal Collaborative Meetings Foster Inter-curricular Teaching and 
Teacher Buy-in to Schoolwide Issues 


Two aspects of the CSRD reform process stand out as instrumental in bringing about 
the impressive changes that have been made by this school. The first is the leadership of 
the principal and assistant principal of the school. The principal contributed to reform by 
empowering staff to take on the reform process and make needed changes. The assistant 
principal worked to provide the tools and opportunities for school staff to take on the 
challenge and to be effective. 


The primary venue for the change process was establishment of the Critical Friends 
Groups (CFG’s), designed to address both individual needs of colleagues as well as school 
wide issues. Over time this interaction produced an environment of trust among colleagues 
that created an environment in which staff were able to take more risks and become more 
creative problem solvers. Change in how the entire school community functions has come 
about over the past three years directly because of the extensive buy-in of staff and 
administrators to make the reform process and associated strategies work. 


The school district has supported CSRD implementation by providing trainers for the 
adopted methods who were able to provide extensive follow-up training at the school site 
on an ongoing basis. The training and support provided to School F was praised by 
teachers and administrators as effective and has led to a considerable change in the school's 
class schedule, curriculum offerings, staff organization and school culture. 


Component 6: Parent and Community Support 


School O: Parents and Community Members Participate in School Governance 
and Goal Setting 


One of the promising practices in School O is the collaboration between the School 
Advisory Council, composed of community and staff members, and the Principals Cabinet 
which includes two administrators, the magnet and SIM coordinators and the lead 
mathematics, science, and reading teachers. The role of the Advisory Council is to develop 
the School Improvement Plan. Based on a review of past progress, they set goals and 
objectives related to student achievement, community involvement and student 
participation in co-curricular programs. The Principals Advisory Cabinet develops the 
specific plans that translate these goals and objectives into a sequence of strategies within a 
time frame. They help to determine what strategies and activities will be employed by 
whom and when. This collaboration between the two groups moves the school 
improvement plan from paper to reality. The key to the success of this process lies in close 
coordination between the two groups so that the Principal's Cabinet fully understands and 
fully implements the school improvement plan. 
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Descriptions of the 
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Strategies Being Used 


Appendix C 


DESCRIPTIONS OF THE 
RESEARCH-BASED METHODS OR STRATEGIES BEING USED 


This appendix contains brief descriptions of the research-based methods in place at 
the 18 CSRD schools visited as part of the Field-Focused Study. The descriptions were 
obtained from The Catalog of School Reform Models, developed and maintained on the 
website for the Northwest Regional Educational Laboratory (www.nwrel.org). 
Descriptions of the locally-developed models are not available through this source but were 
obtained from the models’ originators (or developers). 
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Northwest Regional Educational Laboratory, 10/20/02 


Accelerated Schools (K - 8) 


Accepted for Inclusion 2/1/98 
Re-accepted 11/1/01 
Description Updated 12/1/01 


Type of Model 
Henry Levin, Stanford University 


Current Service Provider National Center for Accelerated Schools Project at the University of Connecticut, and 
various regional centers 


Year Established 1986 
# of Schools Served (9/1/01) 1,300 


ounder 
Primary Goal provide all students with enriched instruction based on entire school community’s vision of 
learning 


Main Features gifted-and-talented instruction for all students through "powerful learning" 
participatory process for whole-school transformation 
three guiding principles (unity of purpose, empowerment plus responsibility, and building 
on strengths) 

Impact on Instruction teachers adapt instructional practices usually reserved for gifted-and-talented children for 
all students 

Impact on Organization/Staffing | governance structure that empowers the whole school community to make key decisions 
based on the Inquiry Process 


Impact on Schedule depends on collective decisions of staff 

Subject-Area Programs no 

Provided by Developer 

Parental Involvement parent and community involvement is built into participatory governance structure 


Technology depends on collective decisions of staff 
Accelerated Schools Resource Guide plus a field guide for each training component 


Origin/Scope 


The accelerated schools approach, developed by Henry Levin of Stanford University, was first implemented 
in 1986 in two San Francisco Bay Area elementary schools. The Accelerated Schools Project has now 
reached over 1,300 schools. 


General Approach 


Many schools serve students in at-risk situations by remediating them, which all too often involves less 
challenging curricula and lowered expectations. Accelerated schools take the opposite approach: they offer 
enriched curricula and instructional programs (the kind traditionally reserved for gifted-and-talented children) 
to all students. Members of the school community work together to transform every classroom into a 
“powerful learning” environment, where students and teachers are encouraged to think creatively, explore 
their interests, and achieve at high levels. 


No single feature makes a school accelerated. Rather, each school community uses the accelerated schools 
process and philosophy to determine its own vision and collaboratively work to achieve its goals. The 
philosophy is based on three democratic principles: unity of purpose, empowerment coupled with 
responsibility, and building on strengths. 
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Transformation into an accelerated school begins with the entire school community examining its present 
situation through a process called taking stock. The school community then forges a shared vision of what it 
wants the school to be. By comparing the vision to its present situation, the school community identifies 
priority challenge areas. Then it sets out to address those areas, working through an accelerated schools 
governance structure and analyzing problems through an Inquiry Process. The Inquiry Process is a systematic 
method that helps school communities clearly understand problems, find and implement solutions, and assess 
results. 


Results 


Two early small-scale evaluations yielded initial evidence of improved achievement, school climate, and 
parent and community involvement in accelerated schools. A 1993 evaluation comparing an accelerated 
school in Texas to a control school revealed that over a two-year period, fifth grade SRA scores in reading, 
language arts, and mathematics at the accelerated school climbed considerably. Over the same period, the 
scores of a control school declined (McCarthy & Still, 1993). In the other study, Metropolitan Achievement 
Test grade-equivalent reading scores at an accelerated school improved more than scores in a control school 
in four of five grades, although the results for language scores were mixed (Knight & Stallings, 1995). 


More recent studies involving larger numbers of elementary schools have also demonstrated gains for 
accelerated schools relative to comparison schools. In an independent study of eight different reform models 
in Memphis, the Accelerated Schools Project was one of three models that demonstrated statistically 
significant or nearly significant growth across all subjects on the TVAAS (Tennessee Value-Added 
Assessment System) compared with control schools. In reading, the Accelerated Schools Project showed the 
highest gain of any model across the three years of the study (Ross, Wang, Sanders, Wright, & Stringfield, 
1999). Unpublished data from 34 elementary schools in Ohio that implemented the Accelerated Schools 
Project in 1997 or before reveal that accelerated schools on average showed greater gains from 1997 to 1999 
in fourth- and sixth-grade reading and mathematics on the Ohio Proficiency Test than the districts in which 
they were located. For schools starting their fifth year or beyond in 1997, the advantages were much larger. 
For example, 12% more students in these accelerated schools scored proficient or advanced on the sixth-grade 
reading test in 1999 than in 1997, compared to a 3% decline for district schools (Report for Ohio Center, 
1999). 


Researchers from the Manpower Demonstration Research Corporation (MDRC) recently completed a five- 
year study of eight accelerated schools. They used third-grade reading and mathematics scores from the three 
years prior to implementation to predict what scores would have been during the following five years with no 
intervention. They then compared these predictions with actual scores to see if the accelerated schools 
approach had any impact. They found little or no impact on test scores during the first three years of 
implementation (when the focus was on reforming school structure and governance), then a gradual increase 
in scores during the fourth and fifth years (when substantial changes in curriculum and instruction were 
taking place). Average scores in the fifth year exceeded predicted scores by seven percentile points in reading 
and eight in mathematics, a statistically significant amount (Bloom et al., 2001). 


To date, no studies have analyzed the impact of the Accelerated Schools Project on middle schools. 
Implementation Assistance 
Project Capacity: The National Center for the Accelerated Schools Project is located at the University 
of Connecticut. There are also 12 regional centers across the country based in universities and state 
departments of education. Across the national and regional centers, the Accelerated Schools Project 


employs 62 full-time and 27 part-time staff. 


Faculty Buy-In: 90% of the school community (all teaching and nonteaching staff plus a representative 
sample of other school community members including parents and district personnel) must agree to 
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transform the school into an accelerated school. Students are also involved in age-appropriate 
discussions during the buy-in process. 


Initial Training: For each accelerated school, the National Center or a regional center trains a five- 
member team comprising the principal, a designated coach (often from the district office), a school 
staff member who will serve as an internal facilitator, and two other school staff members. Training for 
this team involves an intensive five-day summer workshop, two subsequent two-day sessions on 
Inquiry and Powerful Learning, and ongoing mentoring by a center staff member. The coach provides 
two days of training for the entire school staff just before the school year begins. 


Follow-Up Coaching: During the first year of implementation, the coach provides the equivalent of at 
least four additional days of training for all staff. Coaches also spend 25% of their time (generally at 
least one day per week) supporting the school. In the early stages, the coach is more of a trainer, 
introducing the process and guiding school community members through the first steps of 
implementation. In later stages, the coach helps schools evaluate how well the model is working, assists 
in overcoming challenges, and continually reinforces the accelerated schools philosophy to keep 
momentum alive. Additionally, an Accelerated Schools Project staff member visits the school three 
times. During the second and third years of implementation, the five-member school team receives a 
total of nine more days of training. 


Networking: The National Center and regional centers host an annual national conference and regional 
conferences, publish newsletters, support Web sites, and maintain a listserv connecting teachers, 
coaches, and centers via e-mail. Networking opportunities also enable accelerated school communities 
to interact with each other on a regular basis. 

Implementation Review: Continual self-evaluation is part of the process in accelerated schools. To help 


schools gather information, the National Center has developed a comprehensive assessment tool called 
The Tools for Assessing School Progress. 


Costs 

The Accelerated Schools Project (National Center and regional centers) charges approximately $45,000 per 
year for a Basic Partnership Agreement (minimum three-year commitment). This fee varies from state to state 
depending on subsidies and grants provided to the local regional center. The agreement includes, in the first 
year: 


! training of a five-member team including the coach, the principal, and three school staff members 
(excluding travel expenses) 


! training materials, including five copies of the Accelerated Schools Resource Guide 
! three site visits by a project staff member 

!_ technical assistance by phone, fax, and e-mail 

! monthly networking opportunities 

! year-end retreat 

! asubscription to newsletters and the project's electronic network 


In addition, schools and/or districts must provide release time for the entire teaching staff for two days of 
initial training and the equivalent of four days of additional training during the first year. They must also 
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schedule weekly meeting time amounting to about 36 hours per year and cover 25% of the full-time salary 
and benefits of the coach (estimated at $12,000-$20,000 for a coach external to the school). 


Over the next two years schools receive targeted professional development in key components of the model, 
on-going technical assistance, monthly networking opportunities, and one site visit by a project staff member. 
Schools may contract with a center for additional site visits and other services as needed. 


State Standards and Accountability 


The Accelerated Schools Project empowers school communities to determine their own priorities for 
improvement. If the school community determines that aligning instruction with state standards and 
assessments is a priority area, then community members address that area by working through the accelerated 
schools governance structure and Inquiry Process. 


Student Populations 


As part of the catalog Web site search mechanism, each model had an opportunity to apply to be highlighted 
for its efforts in serving selected student populations. The five categories were urban, rural, high poverty, 
English language learners, and special education. To qualify for a category, a model had to demonstrate (a) 
that it included special training, materials, or components focusing on that student population, and (b) that it 
had been implemented in a substantial number of schools serving that population. 


The Accelerated Schools Project is highlighted in all five categories. It was designed primarily to serve 
schools with high proportions of students in at-risk situations. Hundreds of rural and urban schools with large 
concentrations of high poverty students have become accelerated schools. The model provides a process for 
addressing the unique needs of each school, often resulting in special efforts such as tutoring, after-school 
programs, or connections with social service organizations. Training includes strategies for instruction and 
curriculum development within the context of multicultural classrooms. The accelerated schools governance 
model joins special and regular education teachers together in teams, where they work toward the integration 
of special and regular education students. 
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Special Considerations 


The accelerated schools process can be a challenging one. Teachers and administrators must be willing to 
relinquish hierarchical decision-making structures, work together, and expend considerable time and energy 
to transform a traditional school into an accelerated school. Founder Henry Levin estimates that this process 
can take three to five years. During this time, it is crucial to maintain regular meeting time and active 
coaching at the school site. 


Selected Evaluations 


Developer/Implementer 


Knight, S. L., & Stallings, J. A. (1995). The implementation of the accelerated school model in an urban 
elementary school. In R. L. Allington & S. A. Walmsley (Eds.), No quick fix: Rethinking literacy programs in 
America's elementary schools (pp. 236-251). New York: Teachers College Press. 


McCarthy, J., & Still, S. (1993). Hollibrook Accelerated Elementary School. In J. Murphy & P. Hallinger 
(Eds.), Restructuring schooling: Learning from ongoing efforts (pp. 63-83). Newbury Park, CA: Corwin. 


Segal, T. (1999). [Report for Ohio center]. Unpublished raw data. 
Independent Researchers. 


Bloom, H. S., Ham, S., Melton, L., & O’Brien, J., with Doolittle, F. C., & Kagehiro, S. (2001). Evaluating the 
Accelerated Schools Approach: A look at early implementation and impacts on student achievement in eight 
elementary schools. New York: Manpower Demonstration Research Corporation. 


Ross, S. M., Wang, L. W., Sanders, W. L., Wright, S. P., & Stringfield, S. (1999). Two- and three-year 
achievement results on the Tennessee Value-Added Assessment System for restructuring schools in Memphis. 
Memphis: Center for Research in Educational Policy. 


For more information, contact 


Gene Chasin, Director 

National Center for Accelerated Schools Project 
University of Connecticut 

2131 Hillside Road, Unit 3224 

Storrs, CT 06269 

Phone: 860-486-6330 

Fax: 860-486-6348 

E-mail: info@acceleratedschools.net 

Web site: http://www.acceleratedschools.net 
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Excerpt from description of the Behavioral Modification Model,* 2000 


Behavioral Modification Model 


A positive school climate can have substantial positive effects on reductions in substance abuse, anti-social 
behavior and school leaving (e.g., Mayer et all, 1983; Rutter, 1979; and Gottfredsen, 1988; Embry, 1997). 
There are a number of characteristics of schools and school-based programs that seem to be critical for both 
prevention and intervention success, which can be manipulated in “dose” levels for schools and youth who 
need more shown in table form 


Promoting high levels of engagement in academic task and literacy promotion. 


Encouraging high levels of praise or attention by teachers and school staff for attention to task and 
academic productivity, especially for high- risk children. 


Engaging in differential attention from adults to other students and other behavior (DRO = differential 
reinforcement of other behavior) when a child has minor misbehavior rather than attention to 
children’s negative behavior. 


Using daily group activity rewards for teams or classes rather than weekly, monthly or semesterly 
rewards based on individual points. 


Encouraging daily, self-monitoring and posting of academic and behavioral competencies, including 
frequently changed public displays of work. 


Setting up structures (antecedent) that “channel” probability of positive behavior and reduce “down 
time” during transitions. 


Using every day story models to illustrate pro-social behaviors being mastered. 

Using cognitive-behavioral questions and techniques (“Socratic methods” or “brain builders”) to foster 
mastery of emotionally charged events, as well as during routine times to foster perspective taking 
and prediction skills. 

Sending home daily positive home notes to student’s families for positive behavior and achievement, 
linked to rewards at home, plus extensive involvement of pupils in teach of positive skills to 
families. 

Creating many opportunities for students to hold positions of responsibility or job roles each day. 

Using quick daily response-cost, cognitive mediation and over-correction procedures for acts of 
negative behavior instead of delayed consequences, such as referrals to the office or high-intensity 


verbal or physical reprimands. 


Using a common language of belonging to define the school culture, which is used by both adults and 
pupils. 


Creating clarity of rules, including quizzes about them for group reward. 


*A pseudonym has been assigned to this locally-developed model to ensure confidentiality. 


Encouraging frequent art, music performance, drama and best work displays. 
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! Fostering intensive use of the same principles and tools throughout all areas of the school life — 
classroom, lunchroom, recess, bus, after school activities, hallways and other venues, explicitly 
designed to carry from year to year. 


The PAXIS school system is the first approach to create a full system to achieve these results using emerging 
research on prevention from various projects sponsored by the US Centers for Disease Control, the US 
Department of Education and other funding sources. The materials being developed use not only the scientific 
findings but also the field feedback to develop a system that is sensitive to the demands of contemporary 
schools for successful implementation. The new system also packs in procedures to improve literacy and 
achievement, which have been emerging concurrently to the research on problem behavior. This latter 
development is important because of the well-known links to achievement, verbal fluency and anti-social 
behavior. 


The system is a three-step materials process for staff, children and families. All staff members receive the 
same “Successtory” which provides an engaging story model of how all items fit together. This same 
“Successtory” is shared with every family. The result is all parties can start from the same sheet of music, 
understanding emotional, health and academic benefits for children, families and school staff. The Succestory 
is a synthesis of anecdotes and research woven to map a symbolic model of action, showing “coping and 
overcoming of common “yes-buts.” 


Additional support materials are provided for various functions. For example, there is a guide for leaders in 
the school, which provides the details for each “recipe” being implemented. This guide tells the school 
leaders what, when, where and how— including how much paper will be required, who can help with the 
actions, etc. This guide is accompanied by a CD-ROM. 


Every classroom person receives twelve modules, plus a CD-ROM, which has additional linked items by 
developmental level and training support, linked to the World Wide Web. The modules are given out only as 
the school steps through the sequence—keeping down the “two-foot thick materials” shock down. This 
modular approach, tied to support materials for the whole school and families, moves the culture through 
symbolic modeling, role plays, behavioral rehearsal, praxis, rituals, monitoring and celebrations to create 
successful momentum. The modules are organized around a sequence of six principles or tenets: 

1) Commend others; 2) Help each other; 3) Find advice and wisdom; 4) Avoid hurting people; 5) Make 
amends; and 6) Stick with it. 


Volume 1 of the modules assures success in the classroom and main school functions. Volume 2 makes sure 
that the success spreads to all the transitional areas in which so much problem behavior occurs: playground, 
lunchroom, buses, home, neighborhood. 


Every family receives the main story plus a developmentally appropriate book on each principle, which is 
looped into the various modules and CD-ROMS. The booklets (each 24 pages long) are designed to promote 
skills at home and school, that advance academic, social and emotional competencies. The booklets also 
promote literacy, based on studies of emergent literacy. The booklets tie into various protocols at the school 
to advance family involvement. 
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Excerpt from CES National Website — http://www.essentialschools.org/pub/ces_docs/ 
about/about.html 


About the Coalition of Essential Schools 


Vision and Principles 


The Coalition of Essential Schools (CES) is a national network of schools, regional centers, and a national 
office, working to create schools where intellectual excitement animates every child's face, where teachers 
work together to get better at their craft, and where all children flourish, regardless of their gender, race, or 
class. 


CES schools share a common set of beliefs about the purpose and practice of schooling, known as the CES 
Common Principles. Based on decades of research and practice, the principles call for the creation of: 


! Personalized instruction to address individual needs and interests 


! Small schools and classrooms, where teachers and student know each other well and work in an 
atmosphere of trust and high expectations 


! Multiple assessments based on performance of authentic tasks 
! Democratic and equitable school policies and practice 
!_ Close partnerships with the school's community 


The Coalition sees school reform as an inescapably local phenomenon, the outcome of groups of people 
working together, building a shared vision and drawing on the community’s strengths, history, and local 
flavor. The Common Principles are meant to guide the school in setting priorities and designing practice, as 
each school develops its own programs, suited to its particular students, faculty, and community. CES 
regional centers and CES National seek to support schools in this work. 


CES National Programs and Services 


CES National, based in Oakland, California, provides national networking and professional development 
opportunities, conducts research, and advocates for public policies which support our vision of education. 
Here are our core activities: 


Fall Forum. Each year, we convene over 2,000 educators, policy-makers, parents, and students from 
around the world to exchange ideas and practices. 


CES University. An exciting series of institutes in the areas of school design, classroom practice, 
leadership, and community connections, CES University is offered in a number of cities around the 
country in the spring and summer. 


Horace. Our quarterly journal, Horace provides "hands-on" resources and engages readers in examples 
of best practices from around the country. 


CES National Web Site. Including comprehensive links to schools, centers, and publications, the site 


also provides discussion groups and an interactive data base where practitioners deliberate together and 
share resources. 


C-9 


CES National Affiliation. Schools and centers that affiliate with CES National lend their voice to the 
effort to create an educational system that promotes personalization, rigor, and equity. 


Center Director Meetings. CES National convenes the network of center directors regularly for 
professional development and sharing of ideas. 


Research. CES National runs a variety of research activities, including a current partnership with the 
American Institute for Research to study student outcomes in a national sample of CES schools, and 


our annual report of student results, Principles at Work. 


Advocacy. CES National works with the national media and with other educational organizations to 
promote the creation of an environment in which schools designed according to the CES Common 
Principles can thrive. 


For those interested in adopting CES as part of a Comprehensive School Reform Design (CSRD) grant, our 
regional centers offer extensive services. 


Page last updated: May 15, 2002 
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No written materials available; based on descriptions from school staff. 


Comprehensive Reform Model* 


The model included the implementation of state standards, which the school identified as among the first at 
the state level to embrace educational quality reform and “significantly different from other state and national 
models in that they were more project- and performance-oriented.” Other components of the comprehensive 
model included a Freshman Foundation course, intended to help freshman get prepared for high school, a 
flexible block schedule, an interdisciplinary curriculum, whereby teachers of different subjects would have 
common planning time, hold daily team meetings, and coordinate their instructional schedule, the integration 
of school-to-work (school-to-career) topics, the integrated use of technology into the curriculum, especially 
taking advantage of individually-assigned laptop computers, and a vision of the school as having a “global” 
magnet transcending the four separate magnet programs at the school. 


*A pseudonym has been assigned to this locally-developed model to ensure confidentiality. 


Northwest Regional Educational Laboratory, 10/20/02 
Co-nect (K - 12) 


Accepted for Inclusion 2/1/98 
Re-accepted 11/1/01 
Description Updated 4/1/02 


# of Schools Served (2/1/02) 75 


Main Features design-based assistance for comprehensive K-12 school reform 
customized on-line/on-site training and personal support 
project-based learning 
peer and progress review programs 
leadership processes for whole-school technology integration 


Impact on Instruction emphasis on authentic problems, practical applications, and interdisciplinary projects 


Impact on Organization/Staffing | organization of school into small learning communities (“clusters”); full-time facilitator 
preferred 


Impact on Schedule flexible block scheduling; common planning time for teachers 


Subject-Area Programs literacy program for elementary schools 
Provided by Developer 


Parental Involvement parents are encouraged to be more involved in their children’s learning 


Technology significant investment required; schools need computers and Internet access for teachers (at 
least) to make the most of the online products and services; Co-nect does not provide 
equipment 

Materials Schoolwide Action Priorities (printed kits to help schools address schoolwide issues such as 
literacy); online resources including mini-lessons on teaching strategies, database of best 
practices, facilitated learning modules, and searchable selection of curriculum projects tied 
to a custom-built state standards database 


Origin/Scope 


Co-nect was founded in 1992 by members of the Educational Technologies Group at BBN Corporation. As of 
winter 2002, there were 175 Co-nect schools. 


General Approach 


Co-nect helps schools work through a structured process of school improvement, the ultimate focus of which 
is high quality teaching and learning. The model features a nationwide staff of teaching professionals who 
work directly with teachers and administrators within schools and districts. Co-nect’s training and consulting 
services are supported by a suite of diagnostic tools, on-line learning modules, and other teaching resources, 
including a library of best practices. This combination has been designed to help schools meet adequate 
yearly progress goals through sound, sustainable classroom practices. 


Co-nect offers comprehensive schoolwide and districtwide capacity-building programs that include planning 


for continuous improvement, data-driven decision making, alignment strategies such as curriculum mapping, 
technology integration, benchmarking, and leadership training. In addition, Co-nect provides specialized 
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content in project-based learning, authentic assessment, and literacy (for elementary schools). For example, 
Co-nect consultants help teachers working in interdisciplinary teams to guide students through challenging, 
engaging projects that align with state and local standards. 


Co-nect schools are encouraged to have computers in every classroom, with internet capability, thereby 
enabling client schools to access all of the available on-line resources. 


Results 


A national study conducted by independent researchers from Boston College compared achievement gains in 
24 Co-nect schools across the country with gains in demographically similar schools in the same districts. 
Positive changes occurred in most of the Co-nect schools, and four of the schools had consistently higher 
gains than their comparison schools. Overall, however, the researchers concluded that “the Co-nect design 
was associated with similar changes in test scores as were the comparison schools” (Russel & Robinson, 
2000). 


An independent study conducted by researchers at the University of Memphis and University of Tennessee 
found that four Co-nect schools in Memphis showed stronger achievement gains across all subject areas over 
a three-year period (1995-98) on the Tennessee Comprehensive Assessment System than control schools. 
Across all subjects (reading, language, mathematics, science, and social studies), Co-nect schools’ 1995-98 
change was 14.2 points higher than that of control schools on a Cumulative Percent of Norm scale (where a 
score of 100 indicates that a school has made the expected gain for the year). While not statistically 
significant, this advantage was depicted by the researchers as a “moderately large effect size” (Ross, Sanders, 
Stringfield, Wang, & Wright, 1999). A follow-up analysis showed that these Co-nect schools maintained 
“noticeably higher” gain scores than control schools through 1999 (Ross, Sanders, & Wright, 2000). Two 
schools that began using Co-nect in 1997 had not shown a similar advantage, however. 


A third study of Memphis schools compared five Co-nect schools to four control schools during 1999-2000 
on classroom teaching, computer usage, school climate, and student achievement, among other variables. 
Classrooms in Co-nect schools were characterized by greater use of technology, sustained writing, project- 
based learning, independent inquiry, and cooperative learning than classrooms in control schools. Co-nect 
schools were also found to have more positive school climates. Three of the five Co-nect schools were rated 
as higher-implementing by the district office. From 1998 to 2000, these three schools showed greater gains in 
student achievement than state and district schools (Ross & Lowther, 2000). 


According to a study conducted by the evaluation office of the Cincinnati Public Schools, the six Co-nect 
schools in the district showed an improvement in student performance from 1996 to 1999 that exceeded 
district changes over the same period (Lewis & Bartz, 1999). The researchers calculated T-scores for the Ohio 
Proficiency Test in grades four and six in such a way that the district score each year stayed at 50. The Co- 
nect schools’ scores improved from 45.9 in 1996 to 49.8 in 1999. Although they didn’t calculate statistical 
significance because of small sample sizes, the researchers characterized this as a “meaningful improvement.” 


Data from other school districts around the country also show student performance improving in Co-nect 
schools. For example, from 1999 to 2001, the six Co-nect elementary schools in southern Florida 
demonstrated higher achievement gains than the state as a whole on the Florida Comprehensive Achievement 
Tests. In reading, the percentage of fourth-grade students in Co-nect schools scoring at the proficient level or 
higher increased from 44 percent in 1999 to 64 percent in 2001. Over the same period, the average score 
statewide increased from 52 percent to 61 percent. In mathematics, scores in Co-nect schools increased from 
42 percent to 67 percent, again considerably greater than the statewide increase. 


Implementation Assistance 


Project Capacity: Co-nect's headquarters is in Arlington, Massachusetts. Co-nect currently has 108 full- 
time employees, with a small majority based in the field. 
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Faculty Buy-In: Co-nect provides an informational orientation and buy-in process leading to a faculty 
vote. Co-nect strongly recommends at least a 75 percent vote in favor to proceed with adoption. 


Initial Training: Prior to implementation, the school principal attends a two-and-a-half-day Principal 
Summit in Massachusetts, and the school facilitator attends the weeklong School Facilitator Institute, 
also in Massachusetts. 


Follow-Up Coaching: Local school consultants conduct training workshops throughout the year and 
work directly with teams and individuals in the schools. All faculty members attend at least three days 
of training annually. Members of the instructional leadership team have an additional day, and small 
groups of teachers attend specialized sessions. All told, Co-nect consultants spend about two days per 
month on-site. Telephone and e-mail support are also provided by school consultants, area managers in 
the field, and headquarters staff, in addition to online curriculum and technical support. 


Networking: The Co-nect Exchange, the organization’s Web site, delivers professional training for 
teachers and leaders and supports the growth of a collaborative professional community among 
participating schools. The Exchange offers telecollaborative projects, curriculum resources, online 
training modules, and discussion areas. Co-nect Peer Review is a national school visitation program. 
Co-nect also offers an annual conference for teachers and administrators. 


Implementation Review: Co-nect closely monitors and regularly reviews the progress of 
implementation efforts through a series of annual school progress reviews. 


Costs 


A number of factors determine the cost of a standard three-year implementation, including the size and 
location of the school and the number of other Co-nect schools in the area. Typically, the cost is $65,000 per 
year for three years. This figure assumes a school with up to 40 faculty members, partnering with at least four 
other schools in the same region. It covers the following services: 


— Customized professional development, including workshops for principals, 
the school design team, and the full faculty 


— Frequent visits by regional Co-nect school consultants to work directly with 
school faculty members, conduct customized training, and model best teaching practices 


— Customized assistance with initial data gathering, analysis, and planning 
during the first few months of implementation 


In addition, each school must support a full-time school-based facilitator (typically a faculty member) to 
assist with the change process. The school must provide high-speed classroom Internet access for all teachers 
(at least by the end of the first year of implementation) to take advantage of online training and resources. 
Finally, the school must commit to full participation in Co-nect’s national conference, the Critical Friends 
process, training workshops, and other key activities. 


State Standards and Accountability 


A significant amount of the professional development work Co-nect does in schools revolves around 
standards. Co-nect consultants help teachers identify priorities and map effective strategies that address 
standards and speak to students’ interests. Curriculum mapping helps teachers understand the detail and scope 
of what is being taught across subject areas and grade levels, and it creates opportunities for cross- 
grade/cross-discipline teaching and sharing. 


Ca 


Additionally, Co-nect has compiled an online standards database that includes standards from all 32 states 
where the model has been adopted, as well as selected local and national standards. The standards database 
ties directly into the online ProjectBuilder, thus helping teachers create projects aligned to standards. Co-nect 
also maintains a database of existing projects. In the summer of 2002, this database will be tied into the 
standards database, enabling teachers to tailor projects developed in other states to their own state and local 
standards. 


Student Populations 


As part of the catalog Web site search mechanism, each model had an opportunity to apply to be highlighted 
for its efforts in serving selected student populations. The five categories were urban, rural, high poverty, 
English language learners, and special education. To qualify for a category, a model had to demonstrate (a) 
that it included special training, materials, or components focusing on that student population, and (b) that it 
had been implemented in a substantial number of schools serving that population. 


Co-nect is highlighted in four categories: urban, rural, high poverty, and special education. Most Co-nect 
schools are inner city schools with large numbers of students eligible for free/reduced-price lunch. To address 
the needs of these schools, Co-nect promotes the sensible use of technology to help offset the effects of the 
“digital divide.” The model also offers strategies to ensure a safe and nurturing environment for all students. 
Rural schools can take advantage of Co-nect’s distance learning opportunities: the Co-nect Exchange (Web 
site) and videoconferencing facilities. As for special education, Co-nect supports flexible grouping strategies 
for students and teaming of regular and special education teachers. 


Special Considerations 
Technology requirements include computers and high-speed Internet access for all staff. 
Selected Evaluations 


Developer/Implementer 


Lewis, J. L., & Bartz, M. (1999). New American Schools designs: An analysis of program results in district 
schools. Cincinnati, OH: Cincinnati Public Schools. 


Independent Researchers 


Ross, S. M., & Lowther, D. L. (2000). Impacts of the Co-nect school reform design on classroom instruction, 
school climate, and student achievement in inner-city schools. Memphis, TN: University of Memphis, Center 
for Research in Educational Policy. 


Ross, S. M., Sanders, W. L., Stringfield, S., Wang, L. W., & Wright, S. (1999). Two- and three-year 
achievement results on the Tennessee value-added assessment system for restructuring schools in Memphis. 
Memphis, TN: University of Memphis, Center for Research in Educational Policy. 


Ross, S. M., Sanders, W. L., & Wright, S. (2000). Value-added achievement results for two cohorts of Co- 
nect schools in Memphis: 1995-1999 outcomes. Memphis, TN: University of Memphis, Center for Research 
in Educational Policy. 


Russel, M., & Robinson, R. (2000). Co-nect retrospectives outcomes study. Chestnut Hill, MA: Boston 
College, Center for the Study of Testing, Evaluation, and Educational Policy. 


For more information, contact 


Co-nect 

37 Broadway 

Arlington, MA 02474 

Phone: 617-995-3100 or 877-7CONECT 
Fax: 617-955-3103 

E-mail: info@co-nect.net 

Web site: http://www.co-nect.net 
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Northwest Regional Educational Laboratory, 10/20/02 


High Schools That Work (9 - 12) 


Accepted for Inclusion 2/1/98 
Re-accepted 8/1/01 
Description Updated 12/1/01 


entire-school 
Southern Regional Education Board in Atlanta, Georgia 


Current Service SREB 
Provider 


Year Established 1987 


# 


of Schools Served 1,320 


Level 9-12 


to increase the achievement of all students with special emphasis on career-bound students by 
blending the content of traditional college prep studies with quality vocational and technical studies 


Main Features upgraded academic core 


common planning time for teachers to integrate instruction 
higher standards/expectations 


Impact on Instruction |sites are expected to end low-level courses for all students, increase the use of engaging instructional 


strategies, and provide extra help to all students 
sites develop a guidance and advisement system and align with middle schools and postsecondary 


Organization/Staffing _| institutions; more teachers work together; faculties form focus teams 


Impact on Schedule use of larger blocks of instructional time 


S 
Provided by Developer 


ubject-Area Programs |no; however, consultants provide subject-specific workshops customized to school needs 


Parental Involvement {parents are expected to help their children select a four-year program of study that reflects HSTW 


principles 
sites are expected to have access to the Internet and e-mail 
specific materials are suggested to guide schools in making changes 


Origin/Scope 


High Schools That Work (HSTW) is an initiative of the Southern Regional Education Board (SREB) State 
Vocational Education Consortium that began in 1987. More than 1,300 schools are members of the HSTW 
network. 


General Approach 


High Schools That Work is a whole-school, research- and assessment-based reform effort that offers a 
framework of goals and key practices for improving the academic, technical, and intellectual achievement of 
all high school students. HSTW blends traditional college-preparatory content with quality 
technical/vocational studies. It provides technical assistance and staff development focused on techniques and 
strategies such as teamwork, applied learning, and project-based instruction. It also provides a nationally 
recognized yardstick for measuring program effectiveness: the HSTW Assessment, a test based upon the 
National Assessment of Educational Progress. 


HSTW promotes a changed school environment as a context for implementing 10 key practices: (1) high 


expectations; (2) challenging vocational studies; (3) increasing access to academic studies; (4) a program of 
study that includes four years of English, three of math, and three of science; (5) work-based learning; (6) 
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collaboration among academic and vocational teachers; (7) students actively engaged; (8) an individualized 
advising system; (9) extra help; and (10) keeping score (using assessment and evaluation data to foster 
continuous improvement). HSTW sets high expectations, identifies a recommended curriculum to meet the 
expectations, and sets student performance goals benchmarked to the National Assessment of Educational 
Progress (NAEP). 


Three main ideas lay the foundation for HSTW: 


— Academic and vocational teachers, principals, and counselors work 
together to establish unity of vision, a common process for reorganizing 
the school, and a plan for doing so. 


— Teachers and school leaders are empowered to accomplish their goals 
when they share expertise and learn from each other. 


— Assessment, evaluation, and feedback should drive the process and 
implementation of reform. 


The HSTW framework builds support and collaboration among school and district leaders, teachers, students, 
and families for raising expectations for a more challenging and meaningful high school program of study. 
SREB and its partners assist high schools in customizing the HSTW framework into action plans for creating 
more personalized learning environments leading to improved student motivation and performance. 


Results 


All sites are required to participate in the HSTW Assessment. Based on the curriculum frameworks for the 
National Assessment of Educational Progress, the assessment involves achievement tests in reading, 
mathematics, and science for 12th grade students about to complete a vocational or technical concentration. 


The 260 HSTW sites that participated in the assessment in 1994 and 1996 showed significant improvement in 
mean reading and mathematics scores: from 264 to 273 in reading, and from 281 to 285 in mathematics (on a 
scale of 1 to 500). The percentage of career-bound students meeting the HSTW performance goal (279 in 
reading and 295 in mathematics) increased from 33 percent to 43 percent in reading and from 34 percent to 44 
percent in mathematics. Career-bound students who completed the recommended HSTW curriculum or 
completed intellectually challenging assignments scored higher than career-bound students who did not. The 
18 HSTW schools that participated in an advanced integrated learning network, which provided time for 
academic and vocational teams to do collaborative planning, made more progress over the two-year period 
than all HSTW sites (Bottoms, 1997). 


The 1996 and 1998 HSTW Assessment results were analyzed in separate studies by two external evaluators: 
MPR Associates, Inc., and the Research Triangle Institute. MPR researchers reported that the average gain in 
achievement at the 424 HSTW schools that participated in both assessments was 4 points in reading, 13 
points in mathematics, and 9 points in science. They also found that when schools increase the number of 
students completing the HSTW curriculum by 10 percentage points, they are likely to see a gain in 
achievement scores of 10 to 20 points. Other factors, such as increased belief among teachers in students’ 
capacity to complete challenging courses, increased collaboration among academic and vocational teachers, 
and increased guidance and advisement, were also associated with higher achievement gains (Kaufman, 
Bradby, & Teitelbaum, 2000). 


The Research Triangle Institute study (Frome, 2001) involved 393 schools that had collected student test 
scores and student and teacher survey data in 1996 and 1998. This study reported that (a) the percentage of 
students completing a rigorous program of study, (b) the level of implementation of key HSTW practices, and 
(c) the percentage of students reaching HSTW achievement goals in reading, mathematics, and science had all 
risen significantly over the two-year period. The study also found that schools with larger increases in the 
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percentage of seniors who completed the HSTW program of study in each academic area had larger increases 
in the percentage of students who met the achievement goals. Finally, an increase in the use of best 
instructional practices since beginning HSTW was positively related to an increase in the number of students 
meeting the reading achievement goal. 


Information collected through internal case studies, technical assistance visits, and annual progress 
reports suggests that when sites make progress in implementing the key practices, they tend to get the 
following results: improved achievement and higher attendance, graduation, retention, and postsecondary 
attendance rates. Likewise, dropout rates and discipline referrals tend to decline (Bottoms & HSTW staff, 
1996-2000). 


Implementation Assistance 


Project Capacity: HSTW has 26 member states, as well as many other sites nationwide that implement 
the program. HSTW staff members provide services (technical assistance, staff development, and 
assessment) from SREB headquarters in Atlanta. Member states designate a coordinator for HSTW 
sites and encourage networking of HSTW experts within the state. In addition, each HSTW site 
designates a coordinator for activities at the local level. Site coordinators may be full- or part-time or 
receive stipends for their responsibilities. Typical sites include a stipend within their budget for this 
position. 


Faculty Buy-In: In HSTW member states, sites must receive approval to join HSTW from the SREB 
and the state department of education. Sites must also demonstrate that (1) the majority of faculty are 
committed to supporting the HSTW framework; (2) they will conduct at least a five-year school 
improvement plan as detailed by the HSTW program; and (3) they will participate in the HSTW 
assessment program. Sites in non-member states must also demonstrate that two thirds of the faculty 
are committed to supporting the HSTW framework. 


Initial Training: Training includes a two-day site development workshop for the whole faculty; a 
technical assistance visit to establish a baseline for implementation; a minimum of four days of whole- 
school staff development focused on instructional needs; and a three-day retreat for system/school 
leaders. Schools in member states may also participate in state-led HSTW workshops, and all sites are 
encouraged to participate in national, subject-specific conferences. 


Follow-Up Coaching: Sites receive follow-up visits addressing the site action plan. SREB and state 
departments of education (in member states) broker customized technical assistance and training 
services. In year one, sites receive a three-day team technical assistance visit. In years two and three, 
sites receive assistance in using data to update their action plans, plus additional customized assistance 
and training. For schools receiving CSRD grants, first, second, and third-year contracts specify services 
and expectations. 


Networking: HSTW holds an annual national staff development conference, provides teleconferences 
that link developing schools with successful ones, and publishes a quarterly newsletter and a national 
leadership forum for state policy-makers. New sites are continuously encouraged to visit more mature 
sites and share information via a listserv managed by SREB. Other publications aimed at increasing the 
effectiveness of HSTW sites are also available. General information is available on the SREB Web site. 
Implementation Review: SREB collects information from technical assistance visits, a biennial 
assessment, a teacher survey report, and annual progress reports submitted by schools. Schools monitor 
implementation by analysis of data and use of the Benchmarks for New and Mature HSTW Sites. 


Costs 


Three years of HSTW implementation costs $25,000-$35,000 per year. These costs include services such as a 
site development conference, planning, technical assistance visits, staff and curriculum development, training 
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and resource materials, team conference registration, the assessment package, and an evaluative study. Other 
expenses include funds for stipends and substitute teachers, new kinds of curriculum materials, and travel 
expenses for state, regional or national training. 


State Standards and Accountability 


Sites are expected to align curriculum with state and national standards, and to develop K-12 curriculum maps 
and pacing guides that are to be reflected in planning of instructional activities. HSTW provides a facilitator 
to assist faculty in the process of aligning curriculum to state and national standards. 


Student Populations 


As part of the catalog Web site search mechanism, each model had an opportunity to apply to be highlighted 
for its efforts in serving selected student populations. The five categories were urban, rural, high poverty, 
English language learners, and special education. To qualify for a category, a model had to demonstrate (a) 
that it included special training, materials, or components focusing on that student population, and (b) that it 
had been implemented in a substantial number of schools serving that population. 


HSTW is highlighted in two categories, urban and high poverty. The model serves many urban schools and 


schools in high poverty areas. HSTW training includes specific strategies shown by research to be successful 
in urban settings. Additionally, the model provides extra help (tutoring) and guidance for students. 


Special Considerations 


HSTW requires that sites work to replace the general track, raise graduation requirements, participate in the 
HSTW assessment program, develop a site action plan, and use assessment data to update their action plan. 


Selected Evaluations 

Developer/Implementer 

Bottoms, G. (1997, June). The 1996 High Schools That Work assessment: Good news, bad news and hope 
(Research Brief No. 1). Atlanta: Southern Regional Educational Board. (Other HSTW Research Briefs 
provide additional analysis of 1994, 1996, 1998, and 2000 assessments.) 


Bottoms, G., & HSTW Staff. (1996-2000). Case studies (including high schools in DE, GA, KY, MA, NC, 
SC, TX, and WV). Atlanta: Southern Regional Education Board. 
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Independent Researchers 


Kaufman, P., Bradby, D., & Teitelbaum, P. (2000). High Schools That Work and whole school reform: 
Raising academic achievement of vocational completers through the reform of school practice. Berkeley, CA: 
University of California at Berkeley, National Center for Research in Vocational Education. 


Frome, P. (2001). High Schools That Work: Findings from the 1996 and 1998 assessments. Research Triangle 
Park, NC: Research Triangle Institute. 


For more information, contact 


Scott Warren, Director of CSRD 
Southern Regional Education Board 
592 Tenth Street NW 

Atlanta, GA 30318 

Phone: 404-875-9211 

Fax: 404-872-1477 

E-mail: scott.warren@sreb.org 
Web site: http://www.sreb.org 
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Northwest Regional Educational Laboratory, 10/20/02 


Lightspan Achieve Now (K - 6) 


Accepted for Inclusion 9/1/98 
Description Written 10/1/98 
Costs, Number of Schools, Sample Sites, and Contact Information Updated 5/1/01 


Type of Model 
Founder 


Current Service Provider 


Year Established 


# of Schools Served (5/1/01) | 2,841 


Level 
Primary Goal 


Main Features 


Impact on Instruction 


Impact on 
Organization/Staffing 


Impact on Schedule 


Subject-Area Programs 
Provided by Developer 


Parental Involvement 
Technology 
Materials 


Origin/Scope 


to increase time-on-task, promote family involvement in homework, and facilitate mastery 
learning and teaching 


standards-based learning games that support retention and encourage practice for mastery 
family participation in academic lives of children 

PlayStation® game console loaned to families to attach to television 

ongoing professional development for teachers and staff, and workshops for families 


standards-based teaching and learning in class and at home; increased time-on-task; frequent 
monitoring of student progress 


must assign a Lightspan coordinator for each site; family involvement liaison (staff or volunteer) 
desirable 


time required for planning and professional development 


yes (reading, language arts, mathematics) 


program supports learning at home and two-way communication between school and home 
CDs, multi-media computers, digital multiplayers, Internet 


35 CDs for K-2, 36 CDs for 3-4, and 34 CDs for 5-6; teacher guides for each CD; progress 
charts; content correlations; assessment program 


The Lightspan Partnership Inc. was founded in 1993. Lightspan Achieve Now was implemented in 16 schools 
in 1995-96. As of May 2001, 2,841 schools, serving students from a wide range of economic backgrounds, 
had used the model in classrooms and homes. 


General Approach 


Schools and classrooms committed to an aligned instructional program in reading, language arts, and 
mathematics use Lightspan Achieve Now to increase each student’s engaged time-on-task, promote family 
involvement in homework, and create a learning environment designed around mastery learning and teaching. 


The foundation of Lightspan is family involvement and increased learning through after school use of 
instructional video games, aligned with the school’s curriculum, that teach critical targeted skills and 
strategies. Lightspan is centered around discipline-grounded, standards-based, curriculum-driven, interactive 
technologies. In addition, Internet activities facilitate communications, enhance family involvement, and 


make learning fun. 


When a school signs on to use Lightspan, an overall plan aligns achievement goals; teachers, families, and 
staff are trained; and an Education Partnership Consultant from the national staff is assigned to help align the 
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curriculum to the Lightspan program. When the correlation is completed, teachers start to use Lightspan in 
the classroom and as a homework replacement tool. Students are assessed and grouped accordingly, and then 
regrouped, if needed. The classroom teacher introduces a Lightspan game in the classroom. The teacher might 
then send the game home for students to complete over the next few weeks with their families. Families are 
trained so they understand their role and make the necessary commitment to support their child in completing 
homework. 


Results 


To date, no large-scale, systematic evaluations comparing student achievement in Lightspan schools with that 
in control schools have been published. However, Lightspan has contracted with nationally known 
researchers to conduct a rigorous three-year analysis of 22 Lightspan schools, focusing on student 
achievement and other variables. The study will employ an experimental design and incorporate multiple 
measures. 


Preliminary results from these and other smaller-scale evaluations and case studies have yielded evidence of 
improved academic achievement in vocabulary development, reading comprehension, mathematics problem 
solving, and academic growth during summer programs. At Lansdowne Elementary School in Baltimore 
County, Maryland, 34 percent of students in grades K-2 moved from below grade level performance to 
performance at or above grade level versus movement of just 13 percent of students in a matched school, as 
measured by various standardized tests. In Mesa Public Schools (Arizona) during the 1997-98 school year, 
grade one and grade three students learning English as a second language showed significant gains over a 
control group. Students in three Title I schools in Wichita, Kansas, were compared to peers from three 
matched Title I schools within the district. Results from the Metropolitan Achievement Test, 7th Edition, 
showed reliable gains for the Lightspan group at all grades tested. 


RMC Research surveyed over 2,000 families and 269 teachers over two years to measure Lightspan’s impact 
on learning time, family involvement in homework, and student engagement and motivation. Eighty-eight 
percent of families reported that students spent 30 minutes or more per day on Lightspan homework. Seventy- 
two percent reported that time on Lightspan replaced time typically spent on non-educational television and 
video games. Sixty-six percent reported spending 30 minutes or more per day with their children using 
Lightspan. Sixty percent reported that total time spent with their children on schoolwork increased with 
Lightspan. Over 90 percent of teachers reported finding Lightspan useful for providing practice and 
reinforcement, encouraging cooperative learning, and meeting the needs of individual students. 


Implementation Assistance 


Project Capacity: Headquartered in San Diego, California, Lightspan has over 40 Education 
Partnership Consultants throughout the country. This field staff is augmented by a headquarters team of 
three, a fully staffed Product Support desk, and a staff of curriculum experts who produce teachers’ 
guides and national and state correlations. 


Faculty Buy-In: No formal vote is required for schools to start using Lightspan. Schoolwide buy-in is 
achieved as a collaborative process involving the principal as instructional leader, an assigned site 
coordinator (usually the assistant principal), the family involvement coordinator, and grade-level 
curriculum liaisons. 


Initial Training: Training begins with identifying school needs and reviewing the school action plan. It 
includes site coordinator training, curriculum training for grade level liaisons and classroom teachers 
including product exploration, an introduction to family involvement, and implementation strategies 
discussion. Additionally, families are trained before the program is sent home. 


Follow-Up Coaching: During the first year of implementation, the Education Partnership Consultant 
will model integration techniques, assist schools in setting up the home use portion of the program, and 
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develop a plan for follow-on Family Involvement Workshops. Finally, the consultant, in collaboration 
with school staff, conducts regular program review activities to ensure successful implementation. 


Networking: This is facilitated through regular professional development events held year-round, 
throughout the country. Additional networking opportunities are provided through the FLASH 
newsletter and The Lightspan Network Web site. 


Implementation Review: Continual self-evaluation is built into the implementation process. All schools 
participate in the Self-Evaluation Process using tools developed for this purpose by RMC Corporation. 
Most schools also participate in School-Based Action Research using the Action Research Toolkit 
developed for this purpose by Interactive, Inc. 


Costs 


Lightspan is packaged in grade clusters: K-2, 3-4, and 5-6. Schools must buy an Achieve Now school 
package, teacher licenses, and student licenses for each grade cluster. A minimum of nine professional 
development visits is needed in order to ensure a successful Lightspan implementation. 


A $2,000 school package must be purchased in a school’s initial order and can only be purchased once per 
site. This package includes one set of site materials, one Lightspan Desktop Professional Development CD for 
coordinator training, three on-site professional development visits, and access to the Partner Line for 12 
months ($500 per year succeeding the initial 12 months). A $2,650 teacher license must be purchased for each 
teacher using the program. The license includes one grade cluster curriculum license, one set of curriculum 
support and assessment materials, one Lightspan Desktop Professional Development Series, and one on-site 
professional development visit. Finally, a $600 student license must be purchased for each student who will 
use the program at home. If the program is used in an after-school, summer-school, or computer-lab setting, a 
student license is required for each school computer or PlayStation rather than for each student. 


Optional on-line resources are available, including eduTest@School ($2,500 per year subscription), 


eduTest@SchoolPlus ($4,650 per year subscription), and The Lightspan Network ($3,000 per year 
subscription). 


State Standards and Accountability 
(We soon will be providing information on the model’s support for schools’ efforts to meet standards.) 
Student Populations 


Lightspan Achieve Now is designed to increase learning opportunities and enhance achievement for all 
students. It has been successfully implemented in schools with high numbers of at-risk students, including 
Title I and ESL students. The content is full-motion video, completely audio supported, with contextual help. 
Written materials for families are also available in Spanish. 


Special Considerations 
Lightspan Achieve Now is a flexible instructional tool. Changes in teachers’ classroom practice are 


incremental and based on needs identified in the school improvement plan. Lightspan is designed to be woven 
into classroom practice and assigned homework. 
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Selected Evaluations 

Developer/Implementer 

Baltimore County School District. (1997). [Lansdowne Elementary School]. Unpublished raw data. 
Caldwell County School District. (1997). [Gamewell Middle School]. Unpublished raw data. 
Duncanville Independent School District. (1997). [Central Elementary School]. Unpublished raw data. 
Laurens County School District #56. (1997). [Clinton Elementary School]. Unpublished raw data. 
Independent Researchers 


Blanchard, J. (1998). Eisenhower Elementary School, Mesa Unified School District, Mesa, Arizona. 
Unpublished manuscript, Arizona State University, Tempe. 


Godin, K. (1996-97). Lightspan evaluation research. (Available from RMC Research Corporation, 
Portsmouth, NH). 


Shakeshaft, C. (1998). The Lightspan Partnership, Inc. and the home-school connection in Adams County 
School District 50, Westminster, Colorado. Unpublished manuscript, Hofstra University, Department of 
Administration, Policy & Literacy, Hempstead, NY. 


For more information, contact 


Bernice Stafford 

Lightspan Achieve Now 

10140 Campus Point Drive 

San Diego, CA 92121 

Phone: 888-425-5543, ext. 8563 
Fax: 858-824-8001 

E-mail: bstafford@lightspan.com 
Web site: http://www.lightspan.com 
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Excerpt from “A Teacher’s Guide to the [Performance Tasks Model*]”, 1996 


Performance Tasks 


Performance Tasks build on earlier content knowledge, process skills, and work habits and are strategically 
placed in the lesson or unit to enhance learning as the student “pulls it all together.” Such performance tasks 
are not “add-ons” at the end of instruction. They are both an integral part of the learning and an opportunity 
to assess the quality of student performance. When the goal of teaching and learning is knowing and using, 
the performance-based classroom emerges. 


Performance tasks range from short activities taking only a few minutes to projects culminating in polished 
products for audiences in and outside of the classroom. In the beginning, most performance tasks should fall 
on the short end of the continuum. Teachers find that many activities they are already doing can be shaped 
into performance-learning tasks. 


Two initial concerns of teachers moving toward performance-based classrooms include the amount of time 
needed for performance tasks and the subjectivity traditionally associated with teacher assessment and 
assigning “grades.” 


Time 


The initial move to any new method involves an investment in time. The development of performance- 
assessment tasks is no exception. With a little practice, however, teachers find that they can easily and 
quickly develop performance tasks and assessment lists. This process is further simplified as teachers and 
schools begin to collect and maintain lists of generic tasks and assessments that teachers can adapt for 
individual lessons. Teachers find assessment lists a more efficient way of providing feedback to students than 
traditional methods, thus saving time in the long run. Finally, as students work with performance assessment, 
the quality of their work improves, reducing the time teachers must spend assessing and grading student 
work. 


Examples of Performance Tasks 


Performance tasks should be interesting to the student and well connected to the important content, process 
skills, and work habits of the curriculum. Sometimes students can help in constructing these tasks and 
assessment lists. The following are three performance tasks that call for graphs: 


! (Upper Level) Middle or High School 
(Provide the students with a copy of a speeding ticket that shows how the fine is 
determined.) Say to students: “How is the fine for speeding in our state determined? 
Make a graph that shows teenagers in our town how much it will cost them if they are 
caught speeding. Excellent graphs will be displayed in the Driver’s Education 
classroom.” 


! Elementary School 
(At several specified times during the school day, students observe and count, for a set 
length of time, the number of cars and other vehicles going through an intersection 
near the school.) Say to students: “The police department is considering a traffic light 
or a crossing guard at the intersection near your school. Your help is needed to make 
graphs that show how many vehicles go through that intersection at certain times of 
the day. Excellent graphs will be sent to the Chief of Police.” 
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! Primary School 
(In view of the class, place 10 caterpillars in a box. Place a flashlight at one end, 
while darkening the other by folding over the box top.) Say to students: “Do 
caterpillars move more to the light or more to the dark? Make a graph that shows how 
many caterpillars move to the light and how many move to the dark part of the box. 
Your graphs will be displayed at Open House.” 


Performance Task Assessment Lists 


Performance task assessment lists are assessment tools that provide the structure students need to work more 
independently and to encourage them to pay attention to the quality of their work. Assessment lists also 
enable the teacher to efficiently provide students with information on the strengths and weaknesses of their 
work. In creating performance task assessment lists, teachers focus on what students need to know and be 
able to do. One result is that teachers can more consistently and fairly evaluate and grade student work. 
Information from performance task assessment lists also helps students set learning goals and thus helps 
teachers focus subsequent instruction. Parents can also use assessment lists to monitor their student's work 
in school and to help their children check their own work at home. 
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Northwest Regional Educational Laboratory, 10/20/02 


Roots & Wings (PreK - 6) 


Accepted for Inclusion 2/1/98 
Re-accepted 8/1/01 
Description Updated 9/1/01 


Type of Model entire-school 


Primary Goal to ensure that all children learn to read, acquire basic skills in other subjects areas, and build 
problem solving and critical thinking skills 


Main Features research-based curricula in four subjects 
integrated science and social studies program 
cooperative learning 
one-to-one tutoring 
family support team 


Impact on Instruction prescribed curriculum in the areas of literacy, math, and social and scientific problem solving 


Impact on building advisory committee; full-time facilitator; family support team; one-to-one tutoring 


Organization/Staffing 
Impact on Schedule 90-minute reading periods; 75 minutes daily for primary math, 60 for intermediate math 


; , 
Provided by Developer 
Parental Involvement 
p , 


detailed curriculum materials, teachers manuals, and other materials provided for all core 
subjects 


Origin/Scope 


Roots & Wings, created in 1993 by Robert Slavin, Nancy Madden, and a team of developers at Johns 
Hopkins University, is a comprehensive, whole-school reform model designed to boost the basic skills 
achievement of all students while building problem solving skills, creativity, and critical thinking. As of June 
2001, Success for All, the reading component of Roots & Wings, was operating in 1,800 schools. Some 200 
of these schools have added the math, science, and/or social studies components that constitute Roots & 
Wings. 


General Approach 


The purpose of Roots & Wings is to create well-structured curricular and instructional approaches for all core 
academic subjects, prekindergarten to grade six, based on well-evaluated components and well-researched 
principles of instruction, assessment, classroom management, motivation, and professional development. 
Roots & Wings builds on the Success for All program, initiated in 1987, which provides research-based 
curricula for students in reading, writing, and language arts; one-to-one tutoring for primary grade students 
struggling in reading; and extensive family support services (see description of Success for All). To these, 
Roots & Wings adds MathWings and WorldLab. MathWings is based on the National Council of Teachers of 
Mathematics (NCTM) standards, which emphasize problem solving, reasoning, real-world applications, and 
communication. Students work in mixed ability groups, progressing from concrete experience with 
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manipulatives to a more abstract understanding of mathematical concepts. Many MathWings units use works 
of literature to help students explore concepts in meaningful contexts. 


WorldLab is an integrated approach to social studies and science for grades one through five which 
emphasizes group simulations and investigations of real-world problems. For example, students pretend to be 
citizens of a town struggling with environmental issues. This simulation leads them to investigate real 
problems in their own communities. WorldLab is designed to build on knowledge and skills students are 
learning in language arts and mathematics classes. Physical education, music, and visual arts are used to 
enhance WorldLab simulations and investigations. 


Each school has one full-time facilitator to help implement the program, a family support team to foster 
community and parent involvement, and a building advisory team to evaluate the entire school climate and 
advise the principal on general direction and goals. 


Results 


Success for All, the reading/language arts component of Roots & Wings, has been evaluated extensively, with 
statistically significant positive results for program students compared to control students across many 
studies. (See the description of Success for All for more details.) 


Research on the entire Roots & Wings model is neither as extensive nor as rigorous as that on Success for All. 
However, available data do show positive trends for selected Roots & Wings schools. Over the first three 
years of implementation (1993-96), the four pilot Roots & Wings schools in Maryland demonstrated 
substantially greater gains in third and fifth grade on the Maryland School Performance Assessment Program 
(MSPAP) in all six subjects tested (reading, writing, language, math, science, and social studies) than schools 
statewide. After implementation declined over the next two years (the result of reductions in funding and the 
resignation of a supportive superintendent), scores leveled off. Still, over the five year period, model schools 
showed greater gains than schools statewide on every measure except fifth-grade language (Slavin & 
Madden, 2000). Twelve other Roots & Wings schools in five other states have outgained schools statewide on 
state mathematics tests (Madden, Slavin, & Simons, 2000). 


In a study of restructuring schools in Memphis, Tennessee, researchers reported that schools that adopted 
school reform models, including Roots & Wings, demonstrated greater gains on the Tennessee Value-Added 
Assessment System (TVAAS) than non-restructuring schools. Roots & Wings was one of two models overall 
that showed statistically significant effects compared to non-restructuring schools (Ross et al., 2001). 


Implementation Assistance 


Project Capacity: The Success for All Foundation, located in Baltimore, is the national headquarters 
for Roots & Wings. There are also 20 regional centers throughout the U.S. Overall, the foundation 
employs about 240 full-time trainers, including 180 reading trainers, 20 MathWings trainers, 5 
WorldLab trainers, 20 family support trainers, and 15 middle school trainers. There are also 10 part- 
time trainers. 


Faculty Buy-In: At least 80% of a school’s professional staff must vote on a secret ballot to adopt the 
program. 


Initial Training: For each component (Success for All, MathWings, and WorldLab), all teachers 
receive detailed manuals supplemented by three days of training at the beginning of the school year 
provided by Roots & Wings trainers. Schools often phase in the three components, starting with 
Success for All in year one, followed by MathWings in year two and WorldLab in year three. 


Follow-Up Coaching: As noted in the Success for All description, trainers provide at least 26 person- 
days of on-site assistance over the first year of implementation for that component. Follow-up support 
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for the other components is comparable. Trainers make presentations, lead discussions, visit 
classrooms, and work with the building facilitator. The facilitator also organizes informal sessions to 
allow teachers to share problems, suggest changes, and discuss individual children. 


Networking: Conferences are held annually for principals and facilitators to network with those from 
other schools, receive program updates, and share problem-solving strategies. In many parts of the 
country, schools are joining forces with each other to create local support networks, and in some cases 
experienced schools are becoming mentors for new schools. Roots & Wings produces an annual 
newsletter for all its schools, and its Web site contains general program information and research 
articles. 


Implementation Review: As mentioned in the Success for All description, two trainers make three 2- 
day visits to assess the extent of implementation of that component. (These 12 person-days are part of 
the 26 for that component). Implementation visits continue at a lower level after the first year (8 
person-days in year 2, and 6 person-days each year thereafter). The same review schedule holds for 
MathWings and WorldLab as these components are phased in. The review process involves 
interviewing staff, observing classes, examining data, and writing a summary of their findings. Trainers 
also use these opportunities to coach staff and consult with the facilitator. 


Costs 


Sample costs for a school of 500 students (preK-5) typically range from $75,000 to $80,000 for each of three 
years, as reading, math, and social studies/science are phased in. These estimates include training, materials, 
and follow-up visits (including travel costs). Actual costs, which depend on school size, location, specific 
needs (such as bilingual, ESL, or year-round training), and number of schools collaborating in training, are 
calculated for individual schools. Schools also must cover the costs of a full-time facilitator and staff time for 
attending training sessions. Typically, the program is funded by reallocating a school’s current Title I monies, 
often supplemented by other federal or state funds, such as CSRD funds. 


State Standards and Accountability 


Roots & Wings curricula have been matched with state standards and assessments for almost all states. 
Further, modifications to the program have been made to match state standards, assessments, and response 
forms for many states. Documents showing the alignment of Success for All with state standards and 
assessments can be obtained from the Success for All Foundation. 


Student Populations 


As part of the catalog Web site search mechanism, each model had an opportunity to apply to be highlighted 
for its efforts in serving selected student populations. The five categories were urban, rural, high poverty, 
English language learners, and special education. To qualify for a category, a model had to demonstrate (a) 
that it included special training, materials, or components focusing on that student population and (b) that it 
had been implemented in a substantial number of schools serving that population. 


Roots & Wings is highlighted in all five categories. It has been implemented in many schools serving each 
population. The family support team and the promotion of links with social service organizations help support 
disadvantaged students and families. Provisions for distance learning and joint service to multiple schools 
(with consequent fee reductions) facilitate implementation in rural schools. Success for All, the reading 
program, offers numerous components designed to address the needs of urban students, English Language 
Learners, and special education students. See the description of Success for All for more details. 


Special Considerations 
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Teachers must be willing to use detailed curricular materials. The inclusion of students with learning 
problems in regular classrooms is encouraged to the extent possible. Applications for a given school year 
must be filed before May 1 of the preceding school year. 


Selected Evaluations 
Developer/Implementer 


Madden, N. A., Slavin, R. E., & Simons, K. (2000). MathWings: Effects on student performance (Report No. 
39). Baltimore: Johns Hopkins University, Center for Research on the Education of Students Placed at Risk. 


Slavin, R. E., & Madden, N. A. (2000). Roots & Wings: Effects of whole-school reform on student 
achievement. Journal of Education for Students Placed At Risk, 5(1&2), 109-136. 


(See the Success for All description for additional research on that component of the design.) 
Independent Researchers 


Bodilly, S., with Purnell, S., Ramsey, K., & Keith, S. J. (1998). Lessons from New American Schools 
Development Corporation’s demonstration phase. Santa Monica, CA: RAND. 


Ross, S. M., Wang, L. W., Alberg, M., Sanders, W. L., Wright, S. P., & Stringfield, S. (2001, April). Fourth- 
year achievement results on the Tennessee Value-Added Assessment System for restructuring schools in 
Memphis. Paper presented at the annual meeting of the American Educational Research Association, Seattle. 


For more information, contact 


Roots & Wings 

Success For All Foundation 

200 West Towsontown Boulevard 
Baltimore, MD 21204 

Phone: 800-548-4998 

Fax: 410-324-4444 

E-mail: sfainfo@successforall.net 
Web site: http://www.successforall net 
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Excerpt from “CSRD Executive Summary: [Special Literacy Model] * as a CSRD Provider 
Program,” 1999 


Special Literacy Model 


Research Base 


The Project was developed with several theoretical understandings in mind. The following core 
understandings in learning to read as related to reading as language support theoretical perspectives of the 


Project. 


1. 


Reading is a construction of meaning from text. It is an active, cognitive, and 
affective process. Reading is an interactive process between text and reader with 
the ultimate purpose for the reader to construct meaning (Pearson, Roehler, Dole 
& Duffy, 1990; Rosenblatt, 1938/1976, 1978). 


Background knowledge and prior experience are critical to the reading process. 
The importance of prior knowledge in reading has been documented by 
Anderson and Pearson (1984) and Rumelhart (1980). The more students engage 
in reading and writing, the more their prior knowledge increases and thereby 
strengthens their ability to construct meaning when they read (Allington & 
Cunningham, 1996; Sweet, 1993). In addition, learning concepts of print 
enhances one's engagement in the reading process (Clay, 1972). 


Social interaction is essential in learning to read. Vygotsky (1978) emphasizes 
the importance of social interactions in the learning process. A Vygotskian 
perspective emphasizes the notion of the zone of proximal development, 
building on social interactions that support learning with a more knowledgeable 
other to intrapsychological learning which is internalized within the learner. 
Bruner (1975) elaborates on this concept through his description of scaffolding. 


Reading and writing develop together. Reading and constructive processes 
(Pearson & Tierney, 1984; Squire, 1983). Acknowledgment of the reciprocity 
between reading and writing is critical to good first teaching. Writing slows 
down the reading process (Clay, 1991). The most effective literacy programs 
include both reading and writing for developing literacy acquisition. 


The Project is based on reliable research and promotes a balanced literacy approach addressing reading and 
writing through effective, research-based instructional strategies and proven methods for student learning and 
teaching, and school management which include, but are not limited to the following: 


Reading aloud to children (Adams, 1990; Clark, 1976; Cochran-Smith, 1984; Cohen, 1968; 
Durkin, 1966; Fountas & Pinnell, 1996; Goodman, Y., 1984; Green & Harker, 1982; 
Hiebert, 1983; Ninio, 1980; Pappas & Brown, 1987; Schickedanz, 1978; Wells, 1986) 


Phonological and phonemic awareness (Adams, 1990; Beck & Juel, 1995; Juel, 1991; Mann 
1986; Moustafa, 1995, 1997; Morias, Bertelson, Cary, & Alegria, 1986; Pearson, 1991; 
Tumner, Herriman, & Nesdale, 1988; Winner, Landerl, Linortner, & Hummer, 1991) 


* A pseudonym has been assigned to this locally developed model to ensure confidentiality. 
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— Phonics (Adams, 1990; Cunningham, 1990; Freepon & Dahl, 1991; Kasten & Clar, 1989; 
Ribowsky, 1985; Stice & Bertrand, 1990; Sweet, 1993; Weaver, Gillmeister-Krause, & 
Vento-Zogby, 1996) 


— Shared writing (Allen, 1976), interactive writing, and process writing (Graves, 1976) leading 
to independent writing 


— Shared reading (Holdaway, 1979; Fountas & Pinnell, 1996; Martinez & Roser, 1985; Pappas 
& Brown, 1987; Rowe, 1987; Snow, 1983; Sulzby, 1985; Teale & Sulzby, 1986) 


— Guided reading (Clay, 1991A, 1991B; Fountas & Pinnell, 1996; Holdaway, 1979; Lyons, 
Pinnell, & DeFord, 1993; McKenzie, 1986; Meek, 1988; Routman, 1991; Wong, Groth, 
& O’Flahavan, 1994). 


Evaluation 


Evaluation of the Project at has been documented through numerous individual action research projects 
directly related to student literacy achievement. Analysis of these classroom inquiries over a three-year period 
indicate that when teacher researchers compared high-progress, average-progress and low-progress students’ 
literacy growth, 

82% of the students (N=103) who made the greatest gains were identified as low-progress students. The 
reports further document that 100% of all students made literacy gains on one or more of the assessment 
measures used in each classroom inquiry. The effectiveness of this professional development model for 
comprehensive school reform is determined by the positive impact on student achievement. 


Evaluation of the professional development model has been documented through qualitative as well as 
quantitative measures. Evidence of the training of the statewide effort is documented in the K-3 Leadership 
Initiative in Reading and Mathematics Executive Summary to the Board of Elementary and Secondary 
Education (Elliott & Simoneaux, 1998). 


C-33 


Northwest Regional Educational Laboratory, 10/20/02 


Success for All (PreK - 8) 


Accepted for Inclusion 2/1/98 
Re-accepted 8/1/01 
Description Updated 9/1/01 


Type of Model entire-school 


Founder 
Current Service Provider 
Year Established 
# of Schools Served (601) 
Level 
Primary Gea 


Main Features schoolwide reading curriculum 
cooperative learning 
grouping by reading level (reviewed by assessment every 8 weeks) 
tutoring for students in need of extra assistance 
family support team 


Impact on Instruction in reading classes — prescribed curriculum, cooperative learning; other subjects not affected 
(see Roots & Wings for a description of other curricular components that can be added) 


building advisory committee; full-time facilitator; family support team; tutors 
Organization/Staffing 


Impact on Schedule daily 90-minute reading periods; tutoring 


Subject-Area Programs yes (reading) 
Provided by Developer 


Parental Involvement family support team works to increase parental involvement 
Technology 


Materials detailed curriculum materials, teachers manuals, and other materials provided 


Origin/Scope 


Success for All was founded by Robert Slavin, Nancy Madden, and a team of developers from Johns Hopkins 
University. It is now disseminated by the nonprofit Success for All Foundation in Baltimore, directed by the 
founders. The model was first implemented in an elementary school in Baltimore in 1987. The following year 
it expanded to 6 schools (5 in Baltimore and 1 in Philadelphia). By June 2001, it had grown to 1,800 schools. 


General Approach 


Success for All restructures elementary schools (usually high poverty Title I schools) to ensure that every 
child learns to read in the early grades. The idea is to prevent reading problems from appearing in the first 
place and to intervene swiftly and intensively if problems do appear. 


Success for All prescribes specific curricula and instructional strategies for teaching reading, including shared 
story reading, listening comprehension, vocabulary building, sound blending exercises, and writing activities. 
Teachers are provided with detailed materials for use in the classroom. Students often work cooperatively, 
reading to each other and discussing story content and structure. From second through sixth grade, students 
use basals or novels (but not workbooks). All students are required to spend 20 minutes at home each evening 
reading books of their choice. 
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Students are grouped according to reading level for one 90-minute reading period per day. The rest of the day 
they are assigned to regular age-grouped classes. Every eight weeks, teachers assess student progress using 
formal measures of reading comprehension as well as observation and judgment. The assessments determine 
changes in the composition of the reading groups and help identify students in need of extra assistance. Those 
students receive one-on-one tutoring for 20 minutes per day at times other than regular reading or math 
periods. First graders get priority for tutoring. Tutors are generally certified teachers, though well-qualified 
paraprofessionals may tutor children with less severe reading problems. 


Because parental involvement is considered essential to student success, each Success for All school forms a 
Family Support Team, which encourages parents to read to their children, involves parents in school 
activities, and intervenes when problems at home interfere with a child’s progress in school. The operation of 
Success for All is coordinated at each school by a full-time facilitator who helps plan the program and coach 
teachers. Finally, an advisory committee composed of the principal, facilitator, teacher and parent 
representatives, and family support staff meets regularly to review the progress of the program. 


Results 


From the beginning there has been a strong focus in Success for All on research and evaluation. Numerous 
studies conducted by developers and others have compared scores on standardized reading tests (specifically, 
the Durrell Oral Reading Scale and several scales from the Woodcock Reading Mastery Test) for students in 
Success for All schools and control schools. For example, in one study (Madden et al., 1993), students at the 
first five Success for All schools outperformed students at control schools by statistically significant margins 
in every grade. By third grade, the advantage for Success for All students translated into a grade equivalent 
difference of more than eight months. For students in the lowest 25% of their cohorts, the effects were even 
greater. Several other studies (Dianda & Flaherty, 1995; Slavin & Madden, 1999a) have reported that English 
language learners in Success for All elementary schools outperform those in control schools. 


Results have been similar for all but a handful of studies following the same research design. When the results 
of all these studies are combined (involving thousands of students), statistically significant positive effects are 
found for Success for All cohorts at every grade level. By fifth grade, Success for All cohorts score more than 
a year higher on reading measures than control groups (Slavin & Madden, 1999b). 


According to a recent study (Borman & Hewes, 2000), these benefits for students appear to persist beyond 
participation in the program. Students who attended Success for All elementary schools outscored control 
students by a statistically significant margin on the eighth-grade CTBS/4 reading and mathematics tests and 
were less likely to be referred to special education during their middle school years. 


The impact of Success for All has also been measured using statewide assessments. In Indiana, first and 
second grade students at two Success for All schools scored higher on the statewide ISTEP test than control 
students. There was little difference, however, in the scores of third graders on the test (Ross, Smith, & 
Casey, 1997). More recently, the performance of all 111 Success for All schools in Texas was compared to all 
other schools in Texas on TAAS, Texas’s statewide assessment (Hurley, Chamberlain, Slavin, & Madden, 
2000). TAAS reading scores for grades three, four, and five were averaged for all Success for All schools, 
which were divided into cohorts depending on the year of implementation. Gains for each cohort from the 
year prior to implementation to 1998 were compared to gains for the state as a whole over the same period. 
Each Success for All cohort outgained the statewide cohort by at least 4 percentage points. Overall, Success 
for All schools outgained other schools by 5.9 percentage points, a statistically significant difference. 


Success for All recently developed a middle school model, but no evaluations of this model have been 
completed. 
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Implementation Assistance 


Project Capacity: The Success for All Foundation, located in Baltimore, is the model’s national 
headquarters. There are also 20 regional centers throughout the U.S. Overall, the foundation employs 
about 240 full-time trainers, including 180 reading trainers, 20 family support trainers, and 15 middle 
school trainers. The other 25 trainers focus on the mathematics, science, and social studies components 
of Roots & Wings. (See the description of Roots & Wings for more details.) There are also 10 part-time 
trainers. 


Faculty Buy-In: At least 80% of a school’s professional staff must vote on a secret ballot to adopt the 
program. 


Initial Training: In the spring prior to implementation, the school’s principal and designated building 
facilitator attend a week-long training session in their region. In August, project staff members visit the 
school for three days of intensive training for the full school staff, plus a fourth day for tutors. 


Follow-Up Coaching: Over the first year of implementation, trainers provide at least 26 person-days of 
on-site assistance to introduce new components of the program, coach teachers, and work with the 
building facilitator. Over time, the facilitator (a full-time position) assumes most of the coaching and 
problem-solving responsibilities. 


Networking: Success for All supports a Web site, publishes a newsletter, and hosts an annual national 
conference. 


Implementation Review: Three times during the first year, two trainers visit each school for two days 
to assess the extent of implementation. The trainers interview staff, observe classes, examine data, and 
write a summary of their findings. They also use these opportunities to coach staff and consult with the 
facilitator. (These 12 person-days are part of the 26 mentioned above.) Implementation visits continue 
at a lower level after the first year (8 person-days in year 2, and 6 person-days each year thereafter). 


Costs 


Sample costs for a school of 500 students (preK-5) typically range from $75,000 to $80,000 for year one, 
$30,000 to $35,000 for year two, and $23,000 to $25,000 for year three. These estimates include training, 
materials, and follow-up visits (including travel costs). Actual costs, which depend on school size, location, 
specific needs (such as bilingual, ESL, or year-round training), and number of schools collaborating in 
training, are calculated for individual schools. Schools also must cover the costs of a full-time facilitator, staff 
time for attending training sessions, and travel expenses for the principal and facilitator to attend the spring 
training session. Typically, the program is funded by reallocating a school’s current Title I monies, often 
supplemented by other federal or state funds, such as Comprehensive School Reform Demonstration (CSRD) 
or Reading Excellence Act funds. 
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State Standards and Accountability 


Success for All curricula have been matched with state standards and assessments for almost all states. 
Further, modifications to the program have been made to match state standards, assessments, and response 
forms for many states. Documents showing the alignment of Success for All with state standards/assessments 
can be obtained from the Success for All Foundation. 


Student Populations 


As part of the catalog Web site search mechanism, each model had an opportunity to apply to be highlighted 
for its efforts in serving selected student populations. The five categories were urban, rural, high poverty, 
English language learners, and special education. To qualify for a category, a model had to demonstrate (a) 
that it included special training, materials, or components focusing on that student population and (b) that it 
had been implemented in a substantial number of schools serving that population. 


Success for All is highlighted in all five categories. Although designed primarily for inner city schools 
serving large numbers of disadvantaged students, it has been implemented in many rural schools as well. It 
offers a number of features for students in each category: 

Urban: specific curricular materials, such as multicultural materials 

High Poverty: tutoring, family support team, and promotion of links with social service organizations 

Rural: provisions for distance learning and joint service to multiple schools (with consequent fee reductions) 
English Language Learners: Exito Para Todos, a Spanish adaptation of the program for use in bilingual 
programs; additional materials (e.g., vocabulary guides and picture cards) and training in strategies (e.g., total 
physical response) that support English as a Second Language instruction through the sixth grade 

Special Education: a firm policy to keep students with reading problems out of special education, through 


grouping, tutoring, and other early intervention efforts (students who are identified as learning disabled are 
included in regular classrooms to the extent possible) 


Special Considerations 


Reading teachers must be willing to use detailed Success for All materials. The inclusion of students with 
learning problems in regular classrooms is encouraged to the extent possible. Applications for a given school 
year must be filed before May 1 of the preceding school year. 


Selected Evaluations 
Developer/Implementer 


Hurley, E., Chamberlain, A., Slavin, R. E., & Madden, N. A. (2001, June). Effects of Success for All on 
TAAS reading scores: A Texas statewide evaluation. Phi Delta Kappan, 750-756. 


Madden, N. A., Slavin, R. E., Karweit, N. L., Dolan, L. J., & Wasik, B. A. (1993). Success for All: 
Longitudinal effects of a restructuring program for inner-city elementary schools. American Educational 
Research Journal, 30, 123-148. 


Slavin, R. E., & Madden, N. (1999a). Effects of bilingual and English as a Second Language adaptations of 


Success for All on the reading achievement of students acquiring English. Journal of Education for Students 
Placed At Risk, 4(4), 393-416. 


C-37 


Slavin, R. E., & Madden, N. (1999b). Success for All/Roots & Wings: Summary of research on achievement 
outcomes. Baltimore: Johns Hopkins University, Center for Research on the Education of Students Placed at 
Risk. 


Independent Researchers 


Borman, G. D., & Hewes, G. M. (2001). The long-term effects and cost-effectiveness of Success for All. 
Unpublished manuscript. 


Dianda, M. R., & Flaherty, J. F. (1995, April). Effects of Success for All on the reading achievement of first 
graders in California bilingual programs. Paper presented at the annual meeting of the American Educational 
Research Association, San Francisco. 


Ross, S. M., Smith, L. J., & Casey, J. P. (1997). Preventing early school failure: Impacts of Success for All on 
standardized test outcomes, minority group performance, and school effectiveness. Journal of Education for 
Students Placed at Risk, 2(1), 29-53. 


Stringfield, S., Millsap, M. A., Herman, R., Yoder, N., Brigham, N., Nesselfodt, P., Schaffer, E., Karweit, N., 
Levin, M., & Stevens, R. (1997). Urban and suburban/rural special strategies for educating disadvantaged 
children: Final report. Washington, DC: U.S. Department of Education. 


For more information, contact 


Success For All Foundation 

200 West Towsontown Boulevard 
Baltimore, MD 21204-5200 

Phone: 800-548-4998 

Fax: 410-324-4444 

E-mail: sfainfo@successforall.net 
Web site: http://www.successforall net 
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Northwest Regional Educational Laboratory, 10/20/02 
Success-in-the-Making (K - 9) 


Accepted for Inclusion 1/1/99 
Description Written 3/1/99 


[Current Service Provider [NCSLeam SSSCSCSCSCSCSCSCSCSC“C~‘“~“~“~*Y 


# of Schools Served 16,000 schools have used SuccessMaker software 
(1/1/99) 
K-9 


Primary Goal increased achievement in reading, language arts, and mathematics 


Main Features computer-assisted instruction designed to meet individual learning needs 
mastery learning model 
balanced instruction focusing on basic skills and higher-order learning processes 


multiple types of assessment and reporting embedded in the software 


Impact on Instruction data derived from students’ use of software can inform regular classroom instruction 


Impact on site coordinator is recommended 
Organization/Staffing 


Impact on Schedule at least one hour per student per week in both mathematics and reading instruction 


Subject-Area Programs _ | yes (reading, language arts, mathematics) 
Provided by Developer 


Parental Involvement student progress reports and portfolios are shared with parents 


Technology stand-alone computers and peer-to-peer, LAN, and WAN networks; cable and Internet capabilities 
for at-home learning 


Materials over 5,000 hours of instructional material including software, authentic literature, multimedia, 
activities, projects, and other resources; teacher guides 


Origin/Scope 


The Success-in-the-Making approach was developed in 1967 by Patrick Suppes of Stanford University, and 
Mario Zanotti, a nationally renowned psychometrist, based on the belief that the use of technology in the 
classroom can accelerate student learning. Software based on the developers’ approach has served more than 
2 million students in 16,000 schools across the country. 


General Approach 


The core of Success-in-the-Making is the NCS Learn SuccessMaker® software, which provides computer- 
assisted instruction in reading, language arts, and mathematics from kindergarten through ninth grade. 
SuccessMaker adapts curriculum content for each user, evaluates student responses on problems and 
activities, and offers a management system for monitoring student progress. 


Based on the mastery learning model, the software automatically determines each student’s path through the 
material. Students are able to complete increasingly more difficult work, as measured by embedded 


assessments aligned to external testing objectives and state standards. 


Consultants work with local educational leaders to develop implementation plans based on district and site 
goals. Typically, students complete individualized instruction several times a week; teachers then add 
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individual or collaborative lessons and activities relating to classroom learning to achieve greater curriculum 
integration. 


Data derived from student work can help teachers plan and improve both computer-assisted and regular 
classroom instruction. For example, reports show areas where students are having difficulty so that teachers 
can coach students in small groups. Data also can furnish information for program guidance at the school and 
district levels. 


As part of the model’s options, teachers can offer authentic literature, writing tools and process instruction, 
and open-ended tools-based mathematics for all levels. Schools can also provide Spanish-English bilingual 
and ESL content for various levels and components. 


Results 


Using SuccessMaker software to support student learning, multiple schools have documented gains in student 
achievement in reading and mathematics, as evidenced by standardized tests and state proficiency exams. For 
example, 13 schools in New York’s District Six were selected to implement the model, based on low 
performance on the third-grade state-mandated reading test. After implementation, post-test results showed a 
higher percentage of these third-grade students reaching or exceeding the State Reference Point than third- 
graders districtwide. In Landisville, Pennsylvania, longitudinal data on over 500 students using the math 
software, tracked from third to sixth grade, showed the mean percentile of the group rising from the 70th 
percentile in third grade to the 80th percentile in sixth grade, as measured by the California Achievement 
Test. The percentage of students in the lowest quartile dropped from 12 percent to 6 percent, and the 
percentage of students in the top quartile increased from 41 percent to 59 percent. In Fort Worth, Texas, 
students using the software for one year at three schools with schoolwide Title I projects showed significant 
gains on the Texas Assessment of Academic Skills (TAAS). The mean gain from 1996 to 1997 for grades 
four and five was 8.0 Texas Learning Index units. Similar gains were reported for reading. 


Additionally, survey results from multiple school sites indicate that students involved in Success-in-the- 
Making demonstrate an increase in self-esteem and a more positive attitude toward learning. 


Implementation Assistance 


Project Capacity: This model is offered through four regional offices located across the United States, 
with 130 consultants providing professional development. Consultants also can prepare district staff to 
train teachers and support local programs through EdPro certification courses offered several times a 
year. 


Faculty Buy-In: Consultants encourage school and district processes that include teachers in selecting 
the program and making decisions on program options. 


Initial Training: Orientation and planning activities involving administrators or other leaders take a 
minimum of one day. Initial training for all teachers and instructional staff involved with the model 
generally includes three days to introduce content, tools, and basic management system functions; show 
participants self-help resources; and discuss initial program implementation issues, such as enrollment 
and scheduling. 


Follow-Up Coaching: Assistance in generating and interpreting reports is a standard follow-up 
component. Several days of site support are recommended each year for informal coaching and 
training. Consultants model new ways to teach — including multimedia teacher presentations and 
interactive group activities using technology — and share classroom and laboratory/center management 
techniques. 
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Networking: Toll free numbers to reach consultants and technical support, e-mail addresses, program 
newsletters, and events for EdPro “graduates” help educators stay informed. Seminars enable schools to 
share information. Teachers and administrators also can communicate and collaborate through an 
educational Web site. 


Implementation Review: Model guidelines suggest a quarterly review of implementation, including 
review of summarizing reports. This review is usually conducted with the site administrator or 
governance group. 


Costs 


Costs vary depending on the size of the model due to volume discount pricing and the amount of professional 
development desired. Costs for a typical elementary school with computers in the classrooms range from 
$362 to $602 per student for a three-year program (or $121 to $201 per student per year). Lower costs are 
possible if schools have a computer laboratory, which can serve larger numbers of students for a given 
number of computers. Release time and budget for substitutes for two to three days of initial training at the 
beginning of the program and for new teachers in subsequent years also needs to be included. 


State Standards and Accountability 

(We soon will be providing information on the model’s support for schools’ efforts to meet standards.) 
Student Populations 

The program provides instruction for diverse learning needs, including mainstream, gifted, special education, 


ESL, Spanish-English bilingual, and at-risk populations. Adaptive devices serve students who have difficulty 
using standard computer equipment. 


Special Considerations 
Helping administrators and teachers learn new ways of delivering and assessing instruction requires ongoing 


professional development and site support. Each school is advised to plan for a minimum of 15 days of 
professional development over a three-year period. 
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Selected Evaluations 
Developer/Implementer 


1997-98 Duval County CCC implementation overview and summary of findings. (1998). Sunnyvale, CA: 
CCC Research and Measurement Department. 


Zanotti, M. (1997). Fort Worth Title I, 1996-97. Sunnyvale, CA: CCC Research and Measurement 
Department. 


Zanotti, M. (1998). Southfield Public Schools evaluation summary August 1997. Sunnyvale, CA: CCC 
Research and Measurement Department. 


Zanotti, M., & Smith, N. (1995). Effectiveness of the CCC CAI Program: Philadelphia Parochial Schools 
global evaluation for 1994-95. Sunnyvale, CA: CCC Research and Measurement Department. 


Independent Researchers 


Community School District Six Integrated Technology Reading Support Project: First year evaluation report 
1995-96. (1996). New York: Metis Associates. 


Laub, C. M., & Wildasin, R. L. (1998). Student achievement in mathematics and the use of computer-based 
instruction in the Hempfield School District. Landisville, PA: Hempfield School District. 


Second year evaluation report 1996-97. (1998). New York: Metis Associates. 


Underwood, J., with Cavendish, S., Dowling, S., Fogelman, K., & Lawson, T. (1994). Integrated learning 
systems in U.K. Schools: Final report. Leicester, UK: Leicester University, School of Education. 


For more information, contact 


JD Dyas 

NCS Learn 

5421 East Williams Boulevard, Suite 151 
Tucson, AZ 85711 

Phone: 888-627-5327 

Fax: 520-615-7601 

E-mail: paul.dyas@ncslearn.com 

Web site: http://www.ncslearn.com 
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APPENDIX D 


Short Summaries of 18 CSRD Schools 


School A 


Characteristics 


Type of School: Elementary (PreK — 5) 
Population: 537 students 
Ethnicity: 59% minority 


Poverty Rate: 69% free or reduced lunch 
Teachers: 41 (FTEs) 
Principal Turnover: | None 
Superintendent Turnover: Spring 1999, Summer 2002 


1. Research-based Method (Component 1). School A began investigating Accelerated Schools (AS) 
during 1996-97 and 1997-98. In June 1998, 97 percent of the faculty voted to adopt the research method, 
starting with 1998-99. That year, the school failed to receive CSRD funding, which then began in 1999-00, 
which was the second year of its use of AS. School A also uses Lightspan to supplement its reading 
instruction in grades 1-3, with one grade starting in 1999-00 and full coverage planned for 2002-03. 


2. Staff Support (Component 5). Support within the school has been very high. Although 60 percent 
of the students were either ESL or identified as having learning problems, all students were integrated in the 
regular classrooms, working on the same content with a team of teachers. One of the most salient features of 
the school’s success in implementing AS was the leadership of the principal and the buy-in of the staff. The 
principal and teachers developed a collaborative process of problem solving and shared decisionmaking that 
is believed to be central to the success others have noticed at the school. 


3. Leadership Role of the Principal (Component 5). School A’s principal has played a major role in 
leading the reform effort. Besides providing vision and rallying the staff to focus on the improvement effort, 
the principal provided the time and training needed to fully implement AS, and the method became the focal 
point for all of the reform efforts of the school. The principal also was instrumental in finding resources to 
insure full implementation. 


4, Comprehensiveness (Component 2). The focus of the school’s reform was to bring all students up to 
grade level before they proceeded to middle school. The entire school staff underwent extensive AS 
professional development, and a training-of-trainers model also helped to institutionalize the method. But the 
strategies for reaching the goals can be stated in a general manner. Interviews, classroom observation, and 
documents all make clear that the staff understand and strongly support the reform process, including some 
familiarity with CSRD’s nine components. 


5. Support from an External Developer (Component 7). The AS National Center has provided 
extensive onsite training and assistance. In fact, the principal believes that the school has now progressed 
beyond the basic level and has expressed concern over the National Center’s ability to serve the advanced 
needs of the school. As of May 2002, she was still negotiating over the type, amount, and cost of services for 
the 2002-03 school year. Using Title I and CSRD funds, the school also has continued to receive external 
support from two consultants in reading and writing. 


6. Reorganization of the School Day or School Operations (Component 1). The school instituted some 
changes in school operations immediately upon adopting AS. All teachers became members of “cadres,” 
cadre members were assigned roles within the group, and release time was provided for cadre working groups 
and leaders. As of May 2002, the cadres and inquiry process were institutionalized at the school. The cadre 
groups also served as “families,” with the school divided into three such families that served as “schools- 
within-school.” 
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The school also altered its daily class schedule in 2001-02 as a further adaptation to AS. Finally, the 
school created an internal coach position, as required by AS. The coach was responsible for conducting 
inservice training, observing classes, and occasionally serving as a substitute to allow teachers to attend 
training or workshop meetings. 


7. Parental and Community Involvement (Component 6). The school carried out a number of activities 
to gain parental support and participation. School events have been well attended, and parents have expressed 
familiarity with AS when questioned. Parents come to parent-teacher conferences twice a year, and these are 
scheduled from 3 to 9 p.m., also splitting time between the school and the district office to give parents a 
choice of location. Many parents volunteer at the school, serving in capacities from classroom assistants to 
fundraisers. From Spring 2000 to Spring 2001, volunteer hours rose from 1,506 to 3,495, and the site visit 
team observed that most classes had a parent assistant. The school conducts parent surveys, and the responses 
have included many positive comments about the school. 


8. Implementation (Component 1). By 2001-02, AS was being used throughout the school, with 100 
percent of the teachers implementing the method in their classrooms. In all classrooms visited, the site visit 
team saw the use of higher-level thinking techniques, including questioning strategies, as well as evidence of 
student teaming and cooperative learning. During 2001-02, teachers received additional training in the 
Powerful Learning component of AS, and on how to address the “multiple intelligences” of students. School 
A continued to be ready for and wanting even more advanced inquiry method training. 


9. Professional Development (Component 3). During 1999-00, the entire school received extensive 
training in the AS philosophy. For 2000-01, the staff received training in “Powerful Learning.” For 2000-01, 
the staff received further training in successful reading strategies for LEP students. In addition to this 
extensive support, the school had four “banking time” days—regular school days with no students in 
attendance. A good deal of the PD has occurred within and been driven by the school’s cadres and families, 
making PD at the school highly focused. Every teacher interviewed agreed that the quality of PD at the 
school was outstanding. In May 2002, the principal noted that formal PD activities would have to be reduced 
in 2002-03 due to district-wide budget cuts. However, teachers believe they can train and support themselves 
in sustaining the effort. 


10. Other Forms of Support for the Staff (Component 5). While teachers indicate they work harder at 
this school than they did at others, they also have had time to prepare, reflect, and collaborate during the 
school day as well as after hours. Once a month, each family has two hours of planning time, and the cadres 
meet regularly. Teachers feel their hard work is appreciated by the school administration in tangible ways 
and that their efforts are paying off in improved student learning. 


11. Measurable Goals and Benchmarks (Component 4). Beginning with 2001-02, the district no 
longer set district-wide school proficiency levels. Rather, each school was required to prepare a 2001-04 
Educational Plan that addresses the state-mandated proficiency levels by grade and subject. The Plan also 
was to incorporate the plans for federal, state, and other funds received by the school. 


12. District Influences (Other External Conditions). The district has provided CSRD support and also 
has undertaken several compatible initiatives, such as having the school develop benchmarks related to the 
state assessments standards. Initially, however, the principal and many teachers indicated that they received 
little support from the district and got even less recognition for what they had accomplished. The school was 
not adversely affected by the districts shift to neighborhood schools for 2000-01. 


13. State Influences (Other External Conditions). The school has been designated as “in need of 
improvement” in the state’s accountability system, meaning that the school has not met the state’s 
benchmarks. Every year, the school has successfully contested this designation because of its high proportion 
of special students. The state has declared School A to be an exemplary CSRD site. 


14. Evaluation (Component 8). Through its cadres, the school has developed a systematic process for 
analyzing student achievement data. In addition, the school surveyed the staff using the Comprehensive 


School Reform Implementation Survey developed by the district. The school also has used the AS project 
school evaluation questionnaire. 


15. Convergence of Resources and Sustainability (Component 9). The internal coach position was 
only partially funded by CSRD funds, and the principal believed she would be able to absorb the costs within 
her budget after CSRD was to end. School A appears to have coordinated its Title I, bilingual education, and 
other external and internal funds in implementing the research methods (AS and Lightspan). By 2001-02, ED 
had identified School A as a “promising” CSRD site, the state had nominated the school for CSRD exemplary 
status, and the school was serving as a model for area schools implementing AS. The site visit team believes 
strongly that School A will sustain comprehensive reform and implementation of AS well beyond the final 
year of the CSRD grant (2001-02). 
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School B 


Characteristics 


Type of School: — High School (9 — 12) 
Population: 768 students 
Ethnicity: 78% minority 


Poverty Rate: 54% free or reduced lunch 
Teachers: 53 (FTEs) 
Principal Turnover: None 
Superintendent Turnover: © Summer 2001 


1. Research-based Method (Component 1). School B adopted High Schools That Work (HSTW) as its 
CSRD research-based method in 1999-00. HSTW required the school to implement an extensive array of 
activities and therefore served as a framework for reform. An important aspect of HSTW is the required 
involvement of all school members, including students and parents. Another important aspect is a shift to 
block scheduling. However, in spring 2001, the faculty voted to postpone the shift, indicating a significant 
change in the level of support for HSTW. The shift may have been related to reports from others’ experiences 
with block scheduling, including a state report showing that non-block schools had slightly larger increases in 
verbal SAT scores. School B also has implemented a “school-within-a-school” curriculum in its health 
occupations program, which complements several of the nine key HSTW practices. 


2. Staff Support (Component 5). Initially, nearly 1/3 of the staff either quit or retired when it became 
clear that reform was about to occur, and replacing them also was a problem due to a teacher shortage. 
During implementation, teachers indicated that 8 teachers (15 percent of the teachers) had not been involved 
in the reform efforts. The principal believed that about 2/3 of the faculty were truly involved in the reform, 
but the school continued to struggle with the remaining teachers. In the spring of 2001, however, the external 
trainers observed that support for the reform effort appeared to be fading—especially support for the block 
periods—and that some teachers were no longer using HSTW strategies in their classrooms. By 2001-02, 
although all teachers had been involved in some HSTW training, a teacher’s questionnaire showed that 24 of 
52 responses were positive and 21 were negative toward reform. 


3. Leadership Role of the Principal (Component 5). The principal was instrumental in securing 
support and understanding for method during the adoption and initial implementation process. She shifted 
funds to allow as many teachers as possible to participate in training and professional development 
opportunities related to the method. However, beginning in Spring 2001, the principal faced resistance by 
staff to block schedule (a key component of HSTW) as opposition by a few department heads and a critical 
report released by the state. Nevertheless, the principal’s leadership skills were instrumental in securing her 
position as the principal for the newly merged high school in 2003-04. 


4, Comprehensiveness (Component 2). School B had started reform before implementing HSTW. For 
instance, in the fall of 1998 and a year prior to the CSRD grant, School B changed the scheduling for all of its 
9th graders to a block schedule for the four core courses. The scheduling was called a “school-within-a- 
school.” The reform method and the comprehensive plan appear to be well understood by faculty, students, 
and parents. 


5. Support from an External Developer (Component 7). In addition to the professional development 
that has been provided, the developer has made two technical review site visits per year and has held monthly 
phone conferences with mentor site leaders. The CSRD site coordinator appears to have a good working 
relationship with the HSTW contact. The coordinator reported three concrete changes attributable to the 
external support: increased use of project-based learning; constant reminders to raise the bar; and increased 
class offerings by a local university. 
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6. Reorganization of the School Day or School Operations (Component 1). As part of the design of 
HSTW, by 2001-02, most of the “basic-” and “remedial-”level courses had been dropped, and most vocational 
students were intermingled with academically-bound students. However, implementation of the desired block 
scheduling has halted. Although School P (which has had an entire year under a fully blocked schedule) will 
be merging with School B in the fall of 2002, the future of the block scheduling is unknown. 


7. Parental and Community Involvement (Component 6). When the school decided to consider 
adopting a school reform method, the faculty was inundated with input from students, parents, and 
community members, and the faculty created a committee with some of these persons as members. During 
the implementation years, parental involvement was addressed through advisement meetings held at an 
evening event. Parent/student shadowing also gives parents the opportunity to observe the performance and 
behavior of their children. Despite other specific reports of positive experiences with parental and 
community involvement, the staff and administration reported that involvement was limited to a core group of 
parents, and the school had difficulty recruiting parent participation beyond that group. 


8. Implementation (Component 1). According to the HSTW trainers, the school’s implementation has 
been at a fairly high level. For instance, HSTW instructional practices, such as asking students to talk about 
what they know on a topic, having students work in teams and groups, and avoiding the lecture method, have 
been implemented in nearly all of the school’s classrooms. Paired reading and other desirable chair 
arrangements also were observed in the classrooms. 


9. Professional Development (Component 3). During 1999-00, the first year of the CSRD grant, all 
staff participated in five days of HSTW-related professional development. In 2000-01, 24 teachers 
participated in the HSTW three-day summer conference. HSTW professional development directly addresses 
CSRD components 1,2,3,5,6, and 7. To increase the value of the professional development, each teacher 
attending an off-site event was required to bring back something to share with the rest of the faculty. The 
principal noted that teachers making HSTW presentations before their peers seemed to be proud of their 
contribution, and teachers also reported sharing their information with their peers, even meeting before or 
after school to share strategies. Also, as a 2001 co-recipient (with the local feeder middle school) of a Making 
Schools Work (MSW) grant, the school receives additional professional development that is related to the 
HSTW efforts. 


10. Other Forms of Support for the Staff (Component 5). No other forms of support were identified. 


11. Measurable Goals and Benchmarks (Component 4). The school’s leadership team has identified 
HSTW benchmarks, indicators, and evidence for the three-year CSRD period and reviews and revises these 
process benchmarks annually. The team also has been involved in collecting benchmark data, and School B’s 
school improvement plan also has process goals and objectives, such as making schoolwide observations and 
setting goals for teacher and student attendance. Overall, the plan has six goals and an average of nine 
objectives for each. The benchmarks also include extensive coverage of student performance on state and 
district assessments as well as meeting other graduation requirements. Curriculum audit teams related to the 
MSW grant also have conducted assessments and made recommendations to School B regarding the 
attainment of its goals and benchmarks. The school also has adopted a new culture of achievement that 
recognizes high-performing students with tangible rewards, such as being sponsored to go to a local 
amusement park. 


12. District Influences (Other External Conditions). The district has provided staff support for CSRD 
implementation as well as supplemental funds for the professional development related to reform. In 
November 2001, voters approved new funds to renovate school buildings, and the school board also voted to 
merge the district’s two high schools starting in 2003-04. School B’s principal was selected to become the 
principal of the merged school. Details of the merger have still not been worked out, and uncertainty over 
staff assignments and other key issues may shift attention from student achievement to school reorganization. 


13. State Influences (Other External Conditions). No state assessment or policy issues have arisen that 
appear related to the reform. The state has provided routine support for CSRD administration and also is 
sponsoring the CSRD evaluation cite in the next item. 


14. Evaluation (Component 8). The state is conducting its own evaluation through a contract with two 
research firms. The evaluation calls for collecting data from a variety of sources, including classroom 
observations. 


15. Convergence of Resources and Sustainability (Component 9). The CSRD site coordinator is paid 
from the core budget, not CSRD, as almost all of the grant was used for professional development. In 
addition, some (small amounts) of Title II and other state and local funds for professional development are 
used. Besides these two examples, there has been no clear evidence that funds have been redirected or 
converged to support reform. As one result, there appear to be no funding streams available to continue the 
school’s relationship with HSTW following the ending of CSRD. 
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School C 


Characteristics 


Type of School: Elementary (K — 6) 
Population: 559 students 


Ethnicity: 93% minority 
Poverty Rate: 94% free or reduced lunch 
Teachers: 38 (FTEs) 
Principal Turnover: None 
Superintendent Turnover: None 


1. Research-based Method (Component 1). The principal reviewed the district’s set of acceptable 
models and selected Co-NECT to recommend to School C’s faculty. The school had adopted Accelerated 
Schools 12 years earlier and therefore had some elements of reform and of Co-NECT (e.g., participatory 
management structure and orientation toward reform) already in place. CSRD participation began in the 
summer of 1999. 


2. Staff Support (Component 5). Faculty voted unanimously to adopt Co-NECT, but the high turnover 
rate among faculty members meant the support had to be continually regenerated. 


3. Leadership Role of the Principal (Component 5). The principal had started whole school reform in 
1989 by implementing Accelerated Schools. The principal has supported the CSRD research-based strategy 
and uses CSRD to show that reform is part of a national movement, not just a principal’s initiative. 


4. Comprehensiveness (Component 2). School C’s School Improvement Plan (SIP) for 2001-02 lists 
several other instructional programs being used at the school, but Co-NECT is recognized as the only 
comprehensive reform strategy. The SIP is aligned with the district’s goals, and district plans had been 
required by the state, starting in 1998-99, to be aligned with state standards. However, a Co-NECT 
implementation plan also exists and provides guidance to the faculty. 


5. Support from an External Developer (Component 7). The developer supports an on-site consultant 
who works with the school’s Co-NECT facilitator and guides many activities, including making substantive 
presentations at all-school assemblies. 


6. Reorganization of the School Day or School Operations (Component 1). The school has changed its 
internal structure, reorganizing according to “small communities,” establishing a technology infrastructure, 
following a “looping” pattern for teachers and students, and having a design team to implement Co-NECT. 


7. Parental and Community Involvement (Component 6). Few parents are involved in school activities, 
and this is the first challenge mentioned by the school’s faculty. Parent involvement is impeded by the 
school’s isolation due to geographic features and possibly by the fact that many of the students are from 
families with two working parents. 


8. Implementation (Component 1). Some teachers believe that Co-NECT competes with their efforts 
to raise student achievement scores, impeding Co-NECT implementation. Teacher turnover has been nearly 
33 percent during the first two years of CSRD, making retraining a constant activity. Student turnover also 
has been high, but no data were available. 


9. Professional Development (Component 3). Professional development includes training on 
comprehensive planning in relation to the SIP, also showing school staff how to interpret student achievement 
data as part of the planning process. Most of the ongoing professional development, occurring on official 


professional days, appears to be used for Co-NECT training. However, professional development 
requirements are not tied to teacher recertification, and so teachers have less incentive to continue their 
professional development. 


10. Other Forms of Support for the Staff (Component 5). Teachers have a common planning period to 
facilitate cooperative learning. 


11. Measurable Goals and Benchmarks (Component 4). The research-based method includes process 
and outcome benchmarks, which the school monitors. 


12. District Influences (Other External Conditions). District revenues, derived from the sales tax and 
in turn dependent on the tourist industry, will be lower during 2002-03, affecting schools by as yet an 
unpredictable amount. District funds also are being depleted by a new state mandate to increase employee 
benefits, and the school’s share of Title I also will be affected by the district’s opening of a new Title I school. 


13. State Influences (Other External Conditions). The state requires that each school (and, in turn, 
each district) have a single improvement plan that specifically links all funded activities from various sources 
to goals, objectives, benchmarks, and strategies. 


14. Evaluation (Component 8). There is no formal external evaluation, but evaluation is integral to the 
SIP and Co-NECT processes. 


15. Convergence of Resources and Sustainability (Component 9). The SIP embraces the use of funds 
from all funding sources as part of an integrated resource allocation plan. The district’s Title I staff assists 
schools in preparing the SIP and showing how Title I and other funds are coordinated within the plan. 
However, the SIP for 2001-02 still failed to capture all sources of funds and had other problems of 
redundancy. 


School D 


Characteristics 


Type of School: Elementary (K — 6) 
Population: 511 students 
Ethnicity: 88% minority 


Poverty Rate: 90% free or reduced lunch 
Teachers: 27 (FTEs) 
Principal Turnover: None 
Superintendent Turnover: None 


1. Research-based Method (Component 1). School D chose the MathWings (MW) program of Roots & 
Wings for grades 1-5 and Success for All (SFA), English language version only, for grades pre-K through 6 as 
its CSRD method. (CSRD funds supported MW.) The school had no other reform methods, even though the 
district was promoting Open Court in reading and SAXON in mathematics. The school began using a new 
social studies textbook for grades 4-6 in 2001-02 because the state intended to begin testing on social studies 
the following year. 


2. Staff Support (Component 5). The staff had voted to adopt SFA in 1997, and its positive experience 
with SFA starting in 1997-98 led to over 80 percent of the school’s 25 teachers voting to adopt MW in 1999. 
In the early years of SFA, a few teachers left the school because they were reportedly not happy with the 
reading curriculum and strategies. However, by 2000-01, most staff were supportive of both programs. 
According to some teachers, the enthusiasm of the principal was necessary to start the reform process, but 
teachers soon began to see the success of the efforts in improved student achievement. 


3. Leadership Role of the Principal (Component 5). The principal was a critical part of the reform 
efforts at the school. Besides being very supportive of adopting SFA and MW, the principal also identified 
funding mechanisms that allowed staff to be paid stipends to attend weekly grade-level and twice monthly 
component meetings. She also provided a $30 incentive (from state funds) for teachers to make home visits. 


4. Comprehensiveness (Component 2). School D’s end-of-year report annually covered its activities 
and progress in regard to each of the CSRD components. In addition, the district required every school to 
develop a plan to improve performance on the district’s “vital signs” and “puzzle pieces.” Many of the puzzle 
pieces directly paralleled the CSRD components. In addition to describing the activities the school was to 
undertake, the plan also specified the lead responsibility for each activity, the start and end dates, and the 
source of funding. Further, many of the strategies and actions in the plan, such as professional development 
and parent-teacher compacts, were specifically tied to MW. 


5. Support from an External Developer (Component 7). The developer offered extensive training for 
SFA (three days’ inservice) when it began in 1997-98 and for MW (two days’ inservice) in its first year. The 
training focused on curriculum and instructional techniques and not on any other component of reform. For 
MW, the developer customized the training by only introducing a few units of the program at the outset and 
then returning several times over the school year to conduct implementation visits and continue training on 
the later units. All staff indicated a high level of satisfaction with the external developer’s training. Teachers 
commented that the external provider was always available to answer questions. 


6. Reorganization of the School Day or School Operations (Component 1). The school fully 


implemented SFA’s recommended 90-minute block schedule, also assessing and regrouping students every 
eight weeks. MW fell within the 90-minute pattern as well. The MW program also organized students into 
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teams, with the high-performing teams receiving certificates or other small rewards as recognition. For 2000- 
01 and 2001-02, School D had a full-time facilitator for SFA and a half-time facilitator for MW (which had 
been a full-time assignment for the first year). These two staff members provided ongoing training to all 
teachers, including model teaching, one-on-one tutoring, and assistance in assessing and placing students. 


7. Parental and Community Involvement (Component 6). Both SFA and MW require parents’ 
involvement in home instructional activities. Besides teachers’ home visits to advise parents, School D also 
held monthly parent nights that were moderately successful in bringing parents to the school, especially if 
food was served or students were performing. According to the principal, expecting parents to participate 
during the school day was unrealistic, and in 2000-01 only a few parent participated. Some parents also did 
not speak English, posing a further barrier. For 2001-02, the school received about $50,000 in state funds to 
facilitate parent involvement and support and hired an outreach worker. Community involvement expanded 
in 2000-01, with a local church adopting the school and 26 volunteers coming from the church to assist 
teachers one to two times per week and tutoring a total of 54 students. 


8. Implementation (Component 1). Teachers and the principal reported that all teachers were using 
SFA and MW. In 2000-01, some teachers were in their first year and therefore just learning to use MW. 
Overall, the school appeared to be implementing the program well, for a score of 4 on the 5-point scale. As 
for fidelity, the external developer noted that on most SFA items, the school was “mechanically 
implementing” the program. The school had made some modifications to RW with the assistance from the 
external developer and also worked with the developer to improve the coordination between reading and 
listening comprehension. 


9. Professional Development (Component 3). Virtually all professional development was linked to 
MW. The SFA facilitator stated that School D also used the results of the eight-week assessment students to 
“help teachers” direct their attention to professional development needs. The school did not keep 
authoritative records of the staff’s professional development activities. However, the principal estimated that 
the school offered more than 40 hours of professional development each, and that about 70 percent of the staff 
participated at this level. 


Because all other schools in the district were using district-promoted reading and mathematics 
curricula, these other schools were able to participate in district-sponsored professional development. School 
D had to organize its own professional development because it had adopted SFA and MW. According to the 
principal, this isolated the school and its staff from the district in some ways, although it also helped to create 
a more cohesive environment because all training activities took place as a group, and the teachers had to rely 
more heavily on each other. 


10. Other Forms of Support for the Staff (Component 5). (See stipends to attend meetings and for 
home visits, under Item 3 above.) School D also received $26,000 in 1999-2000 under the state’s 
performance program (API). 


11. Measurable Goals and Benchmarks (Component 4). All schools were expected to reach the 
benchmarks set forth by the district’s “vital signs” by 2001, which specified quantitative targets for various 
facets of student performance (e.g., reading readiness, attendance, reading and mathematics proficiency). 
Although each school was at a different starting point, the goal for every school was the same. 
Implementation reviews by the external developer and an annual survey of teachers were used to measure 
fidelity and extent of implementation of RW. 


12. District Influences (Other External Conditions). The district played a major role in selecting the 
CSRD schools, and its strategic plan adopted in February 1998 under a new superintendent has been a driving 


force behind School D’s reform efforts. 


13. State Influences (Other External Conditions). The state is still revising its assessments to make 
them standards-based, and this may affect district and school actions in the future. Also, the state began to 
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implement new promotion and retention policies, focusing attention on interventions for students at-risk of 
retention. 


14. Evaluation (Component 8). As part of School D’s evaluation plan, the district contracted with a 
consultant to conduct surveys and data review but then turned evaluation responsibilities to the school for 
2001-02. The district posed the results of the statement assessment, distributing student-specific reports to 
each teacher and covering two years of data to indicate the change in the student’s performance. Overall, 
student achievement scores were rising, with only 79 students reading at grade level in 1997-98 and 291 at 
grade level in the spring 2001 assessment. 


15. Convergence of Resources and Sustainability (Component 9). Rather than having separate plans 
for each federal and state program, the district required each school to develop a single plan, also providing 
training to the school staff. CSRD’s funds were therefore augmented with funds from other sources. The 
scope of School D’s annual improvement plan, which covered a full range of activities and also linked each 
activity to a specific funding source, suggests that resources were being used in a convergent manner. For 
2001-02, the district moved to complete site-based budgeting for all schools, but the principal felt that no 
funds were left over after covering the costs of teachers and supplies. Given this budget reality, School D is 
concerned about finding a way to continue the reform method. Moreover, the SFA reading curriculum was 
not subjected to state review and the school will not be able to use state funds for SFA once CSRD funding 
ends. 
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School E 


Characteristics 


Type of School: High School (9 — 12) 
Population: ~130 students 


Ethnicity: ~80% minority 
Poverty Rate: 85% free or reduced lunch 
Teachers: 9 (FTEs) 
Principal Turnover: Fall 2001 
Superintendent Turnover: Spring 1999, Summer 2002 


1. Research-based Method (Component 1). School E was selected as a high school, although the site 
visits later determined that it also was connected to a middle school. Beginning in 2001-2002, the high 
school separated from the middle school to become an independent charter high school. The high school 
adopted the Coalition of Essential Schools (CES) as its CSRD research-based method. The high school also 
has adopted or investigated other educational models that are consistent with CSRD and CES. 


2. Staff Support (Component 5). The high school faculty had identified CES as the method of choice 
as part of its CSRD award, which started in 1998-99. During 2001-02, the staff have devoted more time to 
the other methods than to CES. Staff have decided to focus future grant money on a new method—the 
Minnesota New Country School—which is consistent with but different from CES. 


3. Leadership Role of the Principal (Component 5). School E operated with a high level of autonomy 
while connected to the middle school. As an independent charter school, School E is administered 
democratically by the nine faculty members, with no formal principal. In Spring 2002, division among staff 
about the governance and mission of the school threatened progress toward reform goals. 


4. Comprehensiveness (Component 2). The school appears not to have had any vision for 
comprehensive school reform, probably due to its pre-occupation with its own survival, restructuring, and 
transition to becoming a charter school. For instance, the school considered applying for another CSRD grant 
in January 2002 because it is a new school. Though the school found other resources to support new reform 
method and did not seek CSRD funding, this intent, along with other field observations, raises questions 
about the comprehensiveness and extent of reform under the school’s original CSRD award. 


5. Support from an External Developer (Component 7). The school has not had contact with the CES 
developer (and has made no payments for services) for the past two years, although school staff have attended 
annual CES meetings. The developer therefore has been unable to provide an estimate for the fidelity of 
implementation. One senior teacher attending the Nov. 2000 annual conference believed that the school was 
implementing the best CES practices as described at the conference. Further, the staff claim that CES’s nine 
principles continue to shape the school’s approach to learning, emphasizing low student-teacher ratios, 
project-based learning, and student responsibility and mastery. The staff would like to receive more support 
from external organizations, but CES has not been mentioned as one of them. 


6. Reorganization of the School Day or School Operations (Component 1). School E was a combined 
middle and high school (grades 6-12) when applying for the CSRD award. Beginning in 2001-02, the high 
school, which had 168 students, separated from the middle school and opened as a charter school at a new 
location, with an enrollment of 130 students. School staff spent the summer of 2001 moving to a new 
building, recruiting, and dealing with a few troubled students. In fall 2001, the school experienced student 
discipline as a major problem, as well as funding shortages because it had yet to receive its expected state 


charter school grant or its Title I grant. In addition, two key staff who had helped to envision the charter 
school unexpectedly took other jobs at the start of the school year. By January 2002, the school had removed 
three disruptive students and had received the grant funding it expected. 


7. Parental and Community Involvement (Component 6). Active parent involvement had led to the 
initial recommendation in December 2000 for the high school to separate from the middle school. Currently, 
however, there has been virtually no parent involvement, even on the part of the four families involved in 
shaping the charter school. At the same time, the school has developed formal partnerships with a few 
community-based organizations. 


8. Implementation (Component 1). CES-like practices are being implemented to only a mixed extent 
in the classrooms. For instance, some teachers were using an integrated curriculum and attempting to use the 
teacher-as-coach and student-as-worker philosophy, but others were teaching in the more traditional mode. 


9. Professional Development (Component 3). The CES developer did provide professional 
development in the first year of the CSRD award. Since then, however, professional development has been 
limited. The high school staff did not participate in the internal schoolwide professional development 
provided over the 18 months prior to 2001-02, and none of the professional development has been provided 
by CES specialists. Currently, professional development sessions are being used to address more mundane 
administrative issues, such as discipline during lunch. 


10. Other Forms of Support for the Staff (Component 5). The school closes early two days a month for 
staff meetings. However, the limited district and state support for professional development raises questions 
about how it will be financed in the future. 


11. Measurable Goals and Benchmarks (Component 4). Previously, the school used student 
achievement benchmarks that were part of its school accountability reports. However, since becoming a 
charter school, the school’s goals and benchmarks are not clear. Further, there have been no CSRD 
implementation benchmarks or timelines. 


12. District Influences (Other External Conditions). In the spring of 2001, the school board overrode 
the school’s own recommendation to close after three years, opting for the charter school alternative. The 
district has only provided professional development and technical assistance on an “as-needed” basis. 
Significant professional development resources have been decentralized to the schools. At the same time, the 
district is adopting high-stakes assessments for 12th grade graduation. 


13. State Influences (Other External Conditions). The school has placed in the lowest category in the 
state’s accountability system and is still under sanctions and at risk of being closed by the district. State 
revenue caps, along with declining enrollments but increasing costs, have caused every school to cut 
educational programs. 


14. Evaluation (Component 8). An external evaluator was active during the first two CSRD years, but 
the evaluator’s report was submitted to an earlier CSRD facilitator, and the evaluator’s work has had no 
impact on the school. 


15. Convergence of Resources and Sustainability (Component 9). Any previous pattern of 


coordinating funds cannot be applied to the new charter school. The new school’s budget for 2001-02 has 
been underfunded, and how coordination, if any, is working has been unclear. 
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School F 


Characteristics 


Type of School: High School (9 — 12) 
Population: 3,535 students 
Ethnicity: 97% minority 


Poverty Rate: 33% free or reduced lunch 
Teachers: 138 (FTEs) 
Principal Turnover: None 
Superintendent Turnover: Fall 2001 


1. Research-based Method (Component 1). School F uses the ten principles of Coalition of Essential 
Schools, along with a multiplicity of other methods (e.g., High Schools that Work and an NSF Urban 
Systemic Initiative award), all embraced within an overarching school-to-career framework whereby students 
identify possible career pathways that then define needed student portfolios and research projects. The school 
is a large urban high school that receives no Title I funds. 


2. Staff Support (Component 5). At School F, reform efforts had been ongoing prior to 1998. In 
March 1998, 100 percent of the faculty voted to adopt the CES principles, and the CSRD award then started 
in March 1999. At the outset, only about 50 percent of the faculty volunteered to participate in some aspect 
of the effort, but the proportion has steadily increased since then. In April 2001, the faculty decided to apply 
for membership with CES. Only after membership has been attained will a CES external team visit the 
school. 


3. Leadership Role of the Principal (Component 5). The principal and assistant principal have been 
instrumental in leading the reform efforts. They have provided faculty with resources to hold collaboration 
meetings during school hours (e.g., substitute teachers), and have provided opportunities for 14 staff to attend 
leadership training so that they could serve as coaches (facilitators) for the collaboration sessions. 


4. Comprehensiveness (Component 2). The overall design, also reflected in the school’s improvement 
plan, encompasses all nine CSRD components, although parental involvement receives less attention than the 
other components. The faculty collaboration groups and other action research teams serve as planning and 
coordinating processes that link the components to each other and to a vision for school-wide reform. Faculty 
understanding of the overall mission is high and also assessed annually in several surveys. 


5. Support from an External Developer (Component 7). CES provided week-long training described 
under professional development. One school administrator rated the training to be of great benefit to the 
school. Training on other methods also has been provided by developers associated with those other 
methods. However, no assessment on the fidelity of use of CES has been made to date. 


6. Reorganization of the School Day or School Operations, and Extent of Implementation (Component 
1). In September 2000, the school started a two-year HSTW award and began using a new block schedule, 
calling for 90-minute classes. The new schedule also was in keeping with the guiding principles of CES, 
providing extra assistance to students to meet state standards, make-up classes, participate in AP courses, and 
take electives. Starting in 2001-02, the school organized itself into three academies to create smaller learning 
environments, with two new academies scheduled to start over the following two years. 


7. Parental and Community Involvement (Component 6). The school has successfully engaged parents 
and community members to serve on key committees, including the school improvement team. However, 
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gaining more extensive parental involvement remains a difficult task, especially because of the blue-collar 
nature of the school’s community. 


8. Implementation (Component 1). All classrooms appear to be using aspects of the various reform 
methods. Most of the faculty are engaged in a collaboration group that meets monthly to share ideas on 
curriculum and instruction and that is central to the overarching school-to-career framework. 


9. Professional Development (Component 3). The collaborative groups and action teams are the main 
method for delivering professional development. The collaborative groups now engage 145 of the 200 staff, 
with a separate Annenberg grant having been used to train 14 coaches to facilitate the collaborative sessions. 
Training has been offered in relation to the specific methods that the school has adopted, including CES and 
HSTW, but also other specific methods in use at the school. For instance, a week-long CES course has 
proved very helpful in developing a common vision, adjusting to the new schedule, and developing plans of 
action. 


10. Other Forms of Support for the Staff (Component 5). Staff members reported conducting peer 
visits to the classes of fellow collaboration team members, as well as reviewing and sharing lesson plans. The 
collaborative groups have provided teachers the opportunity to learn about areas where they should target 
instruction, based on feedback from other teachers in other subject areas. For example, one science teacher 
noted that students were having difficulty with measurement during laboratory experiments. Discussion of 
this item in the collaborative groups led to a move by math teachers to focus on measurement in subsequent 
lesson plans. 


11. Measurable Goals and Benchmarks (Component 4). The school improvement plan has specific 
timelines and goals that are revised annually. School F has not met the minimum standard in reading on state 
assessments, but did so in both mathematics and writing. The reading performance may be related to the fact 
that 90 percent of the students are Hispanic, although the scores do not include special education or Limited 
English Proficient students. The state’s new report card system will add further visibility to the school’s 
performance and pressure it to improve. 


12. District Influences (Other External Conditions). The district has provided considerable support in 
the form of training and technical assistance for the methods and practices related to CSRD, although there is 
not clear coordination among the district’s evaluation office, CSRD administrative office, or Title I office. 
The training and technical assistance has primarily been related to HSTW and the school-to-career framework; 
a district staff person provides the training and hosts monthly coordination meetings for School F with other 
participating schools in the district. 


13. State Influences (Other External Conditions). State resources are used for inservice related to the 
state’s standards and assessment efforts. The state’s new report card system also will have a serious impact 
on individual schools. 


14. Evaluation (Component 8). The district evaluation office administers a survey to monitor progress 
and impact, but no external evaluation is in progress. The survey only received a 60 percent response by 
December 2001, and the district is still trying to obtain the responses from the other 40 percent. The district 
effort does not include any analysis of student achievement data. 


15. Convergence of Resources and Sustainability (Component 9). The school has had a long history of 
coordinating various sources of external funds. The school is endeavoring to identify new sources of funds to 
replace the CSRD monies, which have largely been used to support the collaborative groups by providing 
teacher substitute days and stipends. The district also has adopted the school-to-career concept, and School F 
may be viewed as a model for other schools, especially in its use of collaborative groups and action teams. 
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School G 


Characteristics 


Type of School: Elementary (K — 6) 
Population: 245 students 
Ethnicity: 47% minority 


Poverty Rate: 94% free or reduced lunch 
Teachers: 17 (FTEs) 
Principal Turnover: Fall 2000 
Superintendent Turnover: None 


1. Research-based Method (Component 1). The Special Literacy Model (SLM) is locally-developed 
and is being implemented at a number of schools in the area. SLM is based on academic research on early 
literacy. School G, with about 250 K-6 students, also is using three other methods as part of the reform 
process, most oriented toward language learning. 


2. Staff Support (Component 5). The principal cites the overwhelming support of the staff, partially 
due to the collaborative process used to select SLM, rather than being dictated by some higher organizational 
level. Along the way, professional development choices also were part of the self-selection process and are 
therefore more widely accepted. The school’s small size, collaborative management style, and history of 
reform all lead to strong support for reform. 


3. Leadership Role of the Principal (Component 5). The current principal began in 2000-01 and is 
enthusiastic about SLM, extending it to grades 4-6 and adding a math literacy component with a staffed 
computer laboratory, all supported through CSRD. 


4, Comprehensiveness (Component 2). The comprehensiveness of the reform effort may be reflected 
by the current principal’s view that “literacy means more than language. It includes math, science, and 
computers, as well as reading.” SLM includes all CSRD components and is aligned with the state’s 
standards. The district-initiated school improvement planning (SIP) process has played a central role in 
establishing the comprehensiveness of the reform effort. Every faculty member signs a page in the plan to 
indicate an understanding of his or her role in implementing the plan. 


5. Support from an External Developer (Component 7). The developer is located at an IHE, 80 miles 
from School G, and comes to the school once or twice a year for inservice sessions. There also is a project 
liaison for schools in the region that are using SLM who also visits the school twice a month. The developer 
rates School G’s fidelity as high. 


6. Reorganization of the School Day or School Operations (Component 1). School G was in the 
process of implementing reform—teacher teams, cross-grade planning groups, and whole-school professional 
development sessions—prior to CSRD. 


7. Parental and Community Involvement (Component 6). The principal requires parents to come to the 
school to pick up and sign for report cards, personally greeting every parent and keeping the school open from 
7a.m. to 7p.m. Other activities also are deliberately schedule for evening hours. Parents appear to be more 
aware of their children’s learning than prior to CSRD, even if they cannot identify specific methods. 


8. Implementation (Component 1). SLM was first implemented in 1999-2000 (the first year of CSRD 


funding), beginning with a two-week summer institute for two teachers at a regional IHE. By 2001-2002, all 
teachers in the school were involved in some aspect of the SLM method, through participation in common 
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planning period meetings, sharing teaching strategies, developing cross-curricular lesson plans and attending 
the summer institute. 


9. Professional Development (Component 3). Central to the reform method is the inservice education 
course credit at the local IHE and the presence of both the IHE service director and an on-site facilitator to 
assist in implementing SLM. Nearly all of School G’s teachers participate in SLM-related professional 
development events. The overwhelming majority of professional development is related to comprehensive 
school reform and is results-based (linking student achievement to professional development). Twice a 
month professional development in math also is presented by a math specialist. 


10. Other Forms of Support for the Staff (Component 5). The principal and teachers share a common 
planning period three times a week. Teachers receive stipends to meet before and after school hours to 
participate in benchmarking and planning, and students also receive rewards and recognition. 


11. Measurable Goals and Benchmarks (Component 4). The benchmarking plan, covering both 
process and outcome measures and including all grades, is central to reform (illustrative example shows both 
input and outcome). Evaluation is a self-assessment process, part of the year-round SIP, rather than an 
external evaluation. The SIP process calls for extensive data collection and review—parent surveys, student 
achievement, and focus group findings. School G’s school performance score (based on several tests) 
declined from 1998-99 to 1999-00 from 63.2 to 54.2 percent. However, it improved dramatically to 75.5 in 
the fall of 2001, recognized by the state as “exemplary academic growth,” even though the school remains 
below the state average. 


12. District Influences (Other External Conditions). In 1999, district staff led workshops to prepare 
schools for developing school improvement plans aligned with budgets and needs assessments. The 2000-01 
plan has three priorities: improving academic performance, continuing professional development, and 
increasing parental involvement and student attendance. The district seeks to align all reform efforts with 
state standards and district goals and is viewed by one state official as one of the most advanced districts in 
complying with state requirements. However, the district also has had a serious fiscal shortfall. 


13. State Influences (Other External Conditions). The state adopted a new accountability and testing 
process, causing some shift from a student- to a test-taking-orientation. The state also has mandated that all 
school efforts be integrated into comprehensive school improvement, reflecting state content standards (e.g., 
plans must reflect state standards). 


14, Evaluation (Component 8). There is no formal external evaluation, but evaluation is integral to the 
SIP process. 


15. Convergence of Resources and Sustainability (Component 9). CSRD funds go almost entirely for 


professional development. Title I also is a major funding source. The principal is confident that new grant 
monies also will support the needed activities in the future. 
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School H 


Characteristics 


Type of School: Elementary (PreK — 5) 
Population: 450 students 
Ethnicity: 97% minority 


Poverty Rate: 53% free or reduced lunch 
Teachers: 31 (FTEs) 
Principal Turnover: None 
Superintendent Turnover: Winter 1999, Winter 2000 


1. Research-based Method (Component 1). School H adopted the reading component of Success for 
All (SFA) as its CSRD method. In 1997-98, the district had asked all schools to choose from among the 
methods endorsed by the New American Schools (NAS), and the school’s principal established a committee 
to review these methods. 


2. Staff Support (Component 5). While no (NAS) method completely aligned with the state testing 
system, more than 85 percent of the faculty voted to adopt SFA (after visiting one or two other schools where 
SFA was being used), because it worked with both phonics and whole language, called for skill-level 
grouping, allowed for accountability, and required 90-minute reading blocks. In all, even after the November 
2001 decision to cease using SFA, there appeared to be genuine support for reform by the staff. Even though 
teachers were reluctant to abandon SFA because they believed SFA made them better teachers, the teachers 
remained concerned that School H was one of the only schools not following the district-developed 
curriculum and could perform poorly on the district’s new interim assessments. 


3. Leadership Role of the Principal (Component 5). Faculty reported that the principal and CIC had 
played a central role in the implementation of SFA. Besides rearranging classroom schedules and providing 
staff with additional planning time, the principal also visited classrooms regularly to show support and 
encourage teachers in their use of SFA. 


4. Comprehensiveness (Component 2). School H had a school improvement plan (SIP) with a 
comprehensive set of goals and strategies, such as improved attendance; increased technology use; increased 
parental involvement; and other specific topics. As a result, the SIP generally aligned with CSRD’s 
components, but the match was not perfect. In general, the operation of the school appeared consistent with 
both the SIP and its original CSRD application. The comprehensiveness of the reform effort appeared to be 
widely understood, even though no one referenced the SIP in interviews. 


5. Support from an External Developer (Component 7). Between the fall of 1998 and the spring of 
2001, the developer made implementation visits twice a year. Opinions differed, however, regarding the 
quality or importance of the assistance: a district administrator claimed that the school and district staff often 
had more knowledge of SFA than did the facilitator (attributing it to demands on SFA due to expansion), but 
the CIC stated that the technical assistance was key to the success of the SFA implementation. Apart from 
the SFA developer, a local university provided student teachers who supported the reform effort by tutoring 
students identified as requiring extra assistance based on SFA and state assessments. 


6. Reorganization of the School Day or School Operations (Component 1). Besides calling for a 90- 
minute block and the grouping of students by reading level across classes and grades, SFA also required the 
appointment of a full-time facilitator, a family support team, and one-to-one tutoring for students needing 
extra attention. In addition, the principal ensured that other authorized school staff, including special 
education teachers, were trained in SFA (avoiding conflict with the union by only using classified teaching 
faculty, and not calling on librarians or other staff with more restrictive contracts). With the additional 


trained staff, the principal was able to reduce class size during the SFA period, thus reflecting another change 
in school operations. 


7. Parental and Community Involvement (Component 6). Staff members reported an increase in 
parental involvement in spending time with their children reading each night. This involvement was 
documented by reading logs kept by the students. A core group of about 30 parents attended monthly 
meetings, and for larger family night affairs about 100-150 parents attended. A “breakfast club” also was 
geared specifically to the reform efforts. Despite these activities, the staff nevertheless believed that parent 
involvement in school activities had declined by the spring of 2002—e.g., only 3-5 parents were participating 
in the breakfast club by that time. Community and business sponsors provided incentives to students making 
the honor roll and donated clothing for students to meet the school’s uniform policy. 


8. Implementation (Component 1). SFA was first implemented during the 1998-99 school year, and all 
classrooms were then using SFA during 2000-01. The developer rated School H’s fidelity in using SFA as 
high. The principal and instructional coordinator said that a major advantage of SFA was that it put all 
teachers on the same page, aligning reading instruction throughout the school. Prior to SFA, such instruction 
had varied among classrooms, both in the time devoted and in the content of the instruction. Teachers agreed 
that SFA had made them all better teachers of reading. 


However, in November 2001 the school voted to shift from SFA to the district-developed reading 
curriculum. The school retained the 90-minute block, and in January 2002 the school officially stopped 
implementing SFA, although SFA materials and instructional strategies continued to be used. The main 
reason for the transition was the school’s goal in aligning its program with the state assessment. The use of 
the district-developed curriculum each day and the decision to stop using SFA suggests that the school was 
not convinced that SFA would fully prepare students for the state assessment, even though the SFA 
Foundation had informed the district that SFA was aligned with the state assessment. In addition, the district 
developed a set of content standards that clearly identified the skills to be mastered by students each week. 
While SFA covered the same topics, their sequence was different from that of the district’s standards, and the 
school believed that these differences would negatively affect student performance on the state assessments 
and on the district’s newly-developed quarterly assessments. Teachers also are accountable for their 
“homeroom” students even though, in the SFA groupings, these students may not be in the teachers’ classes. 


9. Professional Development (Component 3). The staff initially received an extensive 2-day training 
related to SFA, followed by a refresher course during the year. SFA training and instructional practices 
carried over to other subjects, although teachers reported that the training was constantly shifting in response 
to issues raised by the teachers. District-supported professional development also shifted from being 
delivered centrally in 1997-98 to school-based professional development by 1999-2000. By that time, the 
district only provided training to campus instructional coordinators (CICs) who, in tum, provided training to 
the rest of the school staff. The CIC in School H also was the SFA coordinator. The professional developed 
focused directly on reform related topics, such as getting teachers to use the standards, accessing and using 
various resources, and delivering specific reading, mathematics, science, or technology content. 


10. Other Forms of Support for the Staff (Component 5). By reorganizing time schedules, the principal 
created the opportunity for an expanded grade-level collaboration meeting, held during alternate Wednesdays 
for 90 minutes. During this time, teachers took the lead in coordinating the training by grade level and also 
planned their work in a more coordinated fashion. According to the SFA instructional facilitator, these 
meetings were critical to the initial success of SFA at School H. 


11. Measurable Goals and Benchmarks (Component 3). The SIP sets out objectives over a five-year 
period but is an annual document that only describes activities and timelines for the upcoming school year. 
Further, not all objectives in the SIP are easily measurable, and there is no routine assessment to serve as 
feedback to determine whether the goals have been achieved or not. The principal stated that the overall goal 
of the school was to attain the “exemplary” (highest) rating in the state’s ranking system for schools. In 
2001-02, it had received a “recognized” rating. Beginning in 2001-02, the school began using data from the 
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district’s nine-week assessments (aligned with the state assessments) to monitor student achievement and 
progress toward meeting state goals. 


12. District Influences (Other External Conditions). The district played a major role in School H’s 
reform activities. First, the district led to the school to adopt SFA, also supplementing the school’s CSRD 
award to support SFA’s implementation. Later, the district informed schools they would no longer receive 
the supplemental funding if they wanted to continued using SFA, because an analysis had shown that students 
performed comparably whether they had used SFA or the district’s own reading curriculum (which did not 
involve the same level of costs as SFA). In August 2001, the district also released content standards that 
provided teachers with a detailed guide to the skills that students were expected to learn each week. The 
guidance was intended to be aligned with the state assessments. In addition, the district instituted an interim 
assessment system, testing students in their mastery of these skills every nine weeks. The assessment is used 
solely for diagnostic purposes, with the district using a computer-based, commercial system to analyze 
results. The feedback then helps teachers to select the needed scope and sequence to remedy deficiencies. 


13. State Influences (Other External Conditions). The state’s assessment system has been a heavy 
influence over district actions because the assessment scores are widely disseminated and, along with a well- 
defined system of state sanctions and rewards, generates great pressure on schools to perform well on the 
state assessment. 


14. Evaluation (Component 8). The school had no formal evaluation plan in place, and the only 
feedback was contained in the implementation reports by the SFA facilitator. 


15. Convergence of Resources and Sustainability (Component 9). The district and school coordinated 
funds in relation to the school’s SIP. The funds included state, Title I, and other federal funds. CSRD was an 
additional funding source for school reform that was already underway, as the decision to adopt SFA 
preceded the school’s application for CSRD funding. With regard to SFA, despite the transition to the district 
curriculum and the ending of support for the external SFA developer after 2000-01, all teachers interviewed 
indicated that they intended to continue using SFA-related classroom management techniques and 
questioning strategies. 
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School I 


Characteristics 


Type of School: Elementary (PreK — 5) 
Population: ~400 students 
Ethnicity: 55% minority 


Poverty Rate: 61% free or reduced lunch 
Teachers: 34 (FTEs) 
Principal Turnover: None 
Superintendent Turnover: Summer 2002 


1. Research-based Method (Component 1). School I adopted Lightspan Achieve Now as its CSRD 
method. The goal was to enrich reading and math instruction in the school and encourage greater 
involvement of parents in their children’s education at home. The school also was a math and science magnet 
school, with teachers expected to teach science at least 45 minutes a day, and the school expanded from being 
K-3 in 2000-01 to K-5 in 2001-02. The school also was using other reform methods, especially Literacy 
Collaborative (LC) that provided a comprehensive approach to literacy instruction in the primary grades, and 
Reading Recovery for a limited number of first grade students who continued to struggle with reading. 
Finally, the school benefits from having small class sizes, with no classroom having more than 20 students. 


2. Staff Support (Component 5). The principal advocated for the method and made key decisions, such 
as foregoing the purchase of additional Lightspan materials in favor of partially funding the salary of a pre- 
school teacher in CSRD’s third year. The staff appears to have voiced little support for Lightspan, resulting 
in mixed implementation experiences as described later under Item 8. 


3. Leadership Role of the Principal (Component 5). The principal indicated that she, in consultation 
with the assistant principal, represented the leadership core and was the primary decisionmaker for the school. 
However, the principal announced that she would retire at the end of the 2001-02 school year. 


4. Comprehensiveness (Component 2). For 2001-02, the school had two schoolwide plans. One 
covered student achievement goals, and the other summarized a variety of school initiatives. However, 
neither plan offered a comprehensive picture or design for the nine CSRD components. The absence of such 
a plan appears to have resulted in a non-alignment of school elements and lack of comprehensive reform. For 
instance, Lightspan was not aligned with the school curriculum or performance standards. In addition, 
teachers did not share a unified vision of the school, as late as the fall of 2002. Teachers described the school 
as having a “split focus” and a mission that was “way too broad.” 


5. Support from an External Developer (Component 7). The developer conducted a one-day inservice 
for all teachers in 1999-00 and made one site visit in 2000-01. However, Lightspan was not used widely in 
the school during those two years. Because of a shortfall of funds, the school discontinued its relationship 
with the Lightspan developer in 2001-02, relying on the training and support of the one teacher who had 
implemented the method and the assistant principal who also served as the technology coordinator. With 
their assistance the 1st grade teachers implemented Lightspan in January 2002, and most of the 1st grade 
parents were given an orientation. 


6. Reorganization of the School Day or School Operations (Component 1). Student were regrouped 
into heterogeneous and homogeneous groups for LC. However, this did not appreciably change school 
operations and there appear not to have been any other changes accompanying the reform initiatives. Neither 
were there any changes in the organizational or governance structures for the school. 
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7. Parental and Community Involvement (Component 6). Involving parents was a struggle for School 
I, and the assistant principal served half-time as a parent coordinator. However, two specific changes 
occurred in the fall of 2001. The first was the creation of the school’s own Parent-Teacher Organization, 
previously shared with a neighboring elementary school. The second was a decline in parent volunteers. 
However, by the spring of 2002, the principal and assistant principal noted that parent involvement had begun 
to increase again, most likely as the result of the PTO and the school’s focus on “one-to-one” conferencing 
with parents. 


8. Implementation (Component 1). The school began implementing Lightspan in 1997-98, one year 
before its start of the CSRD grant. With CSRD, School I increased the number of PlayStations from 
“several” to 90 units by 2001-02—enough to place stations in all K-2 classrooms and send stations home to 
all eligible 1st graders (about 80 percent of the 1st grade students’ parents came to training and signed the 
necessary permission slip). The majority of the CSRD funds, however, were used to support half the salary of 
the assistant principal, to serve as a parent coordinator. Lightspan in 2000-01 was not used in the way 
envisioned. Only one of the 2nd grade teachers integrated the units into her lessons and used their 
assessments. Another teacher that had helped to pilot Lightspan in 1997-98 admitted that she did not use it at 
all, and the other 2nd grade teachers only used the program as an enrichment activity and only because “they 
have to.” Teachers also cited difficulties in getting parents to come for training and in students returning the 
CDS ina timely fashion. However, by 2001-02, the decision to deploy Lightspan in the 1st grade had led to 
improved implementation, with the 1st grade teachers expressing enthusiasm for the program, 80 percent of 
the students using the stations at home and regularly checking out the CDS, and the program integrated into 
the 1st grade “centers.” Nevertheless, because these experiences were largely limited to the 1st grade and 
most teachers were either not using Lightspan or using it as a supplemental activity for students who finished 
in-class assignments early, the program was judged only to be partially implemented. The developer 
provided no formal assessment of the fidelity of the implementation. 


9. Professional Development (Component 3). The general consensus by the school’s staff was that 
there are five inservice days each year, an amount that has not changed since CSRD. The professional 
development was to be related directly to the student achievement and parental involvement goals outlined in 
the school improvement plan. However, a number of teachers suggested that the effectiveness of the 
professional development was minimal. A small amount of continuing education (six units every 5 years) 
was required by the district but not defined and its impact considered insignificant. During CSRD’s first two 
years, teachers commented that the school’s inservice training was aligned, but too many topics were covered, 
and the focus was too broad. 


10. Other Forms of Support for the Staff (Component 5). During the first two CSRD years, teachers 
complained they had insufficient planning time. They also wanted more time for collaborative planning, to 
learn from one another. In response, during 2001-02, School I freed time for such collaboration. Every 
Monday, the specialist teachers oversee all children having a reading session so that the other teachers can 
meet by grade level, share ideas, examine achievement data, and develop plans for improving scores. The 
school’s administration cited this change as a significant factor in increasing implementation, enhancing the 
professionalism of the school’s relatively young staff, and improving school performance. 


11. Measurable Goals and Benchmarks (Component 4). School I had no implementation benchmarks 
for school reform or even for Lightspan. Although the school has student achievement goals, the district 
assessment tool was still changing from 2000-01 to 2001-02, and neither the district nor the state had 
incentives or punitive measures in place for school performance on their assessments. 


12. District Influences (Other External Conditions). The district has been influential in a number of 
ways. It recruited School I into CSRD, helped the school to implement LC and the various initiatives, and 
also helped to integrate the many components. The district also provided achievement data to each school in 
the fall of 2001, disaggregated by child, and the district representative helped School to analyze the data and 
develop a schoolwide plan. The district also provides math and reading objectives to the schools and has 
been working on aligning its criterion-referenced test with those objectives. Finally, the district has helped 
schools in preparing for the state assessments and understanding how to use the assessments to improve 


D-22 


instruction. District budget cuts led to the elimination of its evaluation unit three years ago and also have 
negatively impacted the school’s budget. 


13. State Influences (Other External Conditions). The main state influence has been the severe budget 
cuts affecting all districts and schools in the state. 


14. Evaluation (Component 8). The school had no formal evaluation plan in place, either for 
Lightspan or its overall comprehensive reform effort. 


15. Convergence of Resources and Sustainability (Component 9). Since one of the objectives for 
Lightspan was to increase parental involvement, funds budgeted for a parent coordinator and parent 
education, including the use of some Title I funds ($11,000) were aligned. At the same time, the school 
prides itself on its entrepreneurship and ability to obtain grants from the community and other external 
sources, but it has no plan describing the integration of all the resources in a comprehensive manner. Some 
interviewees suggest that the grants hinder, rather than facilitate comprehensive school reform. According to 
one district representative, “the school has bought solutions before analyzing their problems.” At the same 
time, the principal says she tries to coordinate all programs to meet the single goal of the school, which is to 
close the achievement gap between high and low performers. 


The prospect for sustaining comprehensive reform and the use of Lightspan is unclear. Although 
School I’s reform efforts could be described as unfocused and somewhat random, teachers and administrators 
expressed hope by the spring of 2002 that improvement had begun to occur. They pointed to the successful 
implementation of the 1st grade use of Lightspan, in the absence of external support, as an indication that the 
effort could continue, although the preferences of the new principal for 2002-03 were unknown. 


The school has continued to use LC which also is coordinated with RR, receiving extensive external 
assistance annually from the developer and having a trained LC coordinator, with a second teacher being 
trained. By the spring of 2002, the school believed that LC was starting to pay dividends, with student 
achievement beginning to improve. At that time, teachers also expressed strong support for the method and 
indicated that it was bringing a focus and new energy to the school. 
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School J 


Characteristics 


Type of School: Elementary (K — 6) 
Population: 668 students 
Ethnicity: 95% minority 


Poverty Rate: NA 
Teachers: 49 (FTEs) 
Principal Turnover: Summer 2002 
Superintendent Turnover: Summer 2000 


1. Research-based Method (Component 1). School J chose a locally developed reform practice (called 
Performance Tasks, PT, for the purposes of this report. PT is an approach to teaching and learning that 
embeds performance tasks, but not a specific curriculum, in day-to-day instruction in a variety of subjects and 
across all grades. The tasks are available via a Web site, for teachers to select and use. Although tasks are 
available for all subjects, more are available for core literacy and writing than the other subjects. The 
collection of tasks is continually expanded, as teachers and staff may create new tasks for editing and 
approval. (The reviewers judge whether a task meets district and state standards.) The tasks are to be 
examples of real-life experiences (“authentic”), enabling students to acquire content knowledge, good work 
habits, and thinking and communication skills. Examples include: organizing a healthy luncheon, guiding 
classmates on a trip, keeping a science journal, interviewing family members to write a biographical profile of 
a grandparent, and assessing classroom lighting to advise school staff on ways to conserve energy. Use of the 
tasks as culminating activities often requires teachers to do “backward planning.” Students build portfolios 
and their own self-assessments and carry them onto the next grade, providing a sense of accomplishment and 
ownership over their work. 


School J also had other complementary research-based methods, including an earlier Carnegie 
Corporation’s Middle Grade School Initiative, the TERK math program, and the Character Counts program. 
In addition, a new superintendent hired as School J was entering its second CSRD year (2000-01) had all 
schools adopt IFL’s nine principles of learning. All of the various methods appear consistent with PT. 


2. Staff Support (Component 5). More than 80 percent of the teachers and staff voted to implement PT. 
Prior to the start of CSRD (1999-00), several of the teachers had visited other regional schools where the 
method was already been being used, and they also learned that these schools’ test scores had improved after 
their implementation of PT. By 2001-02, the major support for the method came from the core steering 
committee. Other persons providing strong support have been the home school coordinator and the 
community liaison. The staff seem dedicated to the school and the children’s academic achievement, and the 
school has had a very low teacher turnover rate. 


3. Leadership Role of the Principal (Component 5). The principal is a legendary figure in the school 
district and became a “community hero.” Brought to the district to make radical changes, she reportedly 
“turned the school around.” Central office staff and teachers tell stories about how she freed the school from 
the harassment of drug peddlers and created a safe environment for children. She was a leader in the decision 
to select PT but planned to retire in June 2002. By spring 2002, a new principal had not been named. 


4. Comprehensiveness (Component 2). PT includes all nine CSRD components, although not all have 
been successfully implemented by School J. The school has a core steering committee managing the 
implementation and integration of reform components, and the district staff also supported the school’s 
reform efforts, facilitating efforts to synchronize the school’s actions with the CSRD components (however, 
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the school’s strategic improvement plan does not address the components). Most teachers have indicated 
their support for the reform effort and believe it can make a difference in student learning and performance. 


5. Support from an External Developer (Component 7). External support has been provided by the 
developer of PT in the first year and then by a consultant who worked with the developer in years-2 and -3. 
An external evaluator also has provided assistance, for instance by developing a new assessment and student 
form to give feedback to teachers on their implementation of PT. During the first year, the developer 
provided a total of 40 hours of training to School J’s staff. For the first time in 2001-02, the external 
consultant’s training emphasized the development of new performance tasks, and all teachers in training were 
to develop a task. By fall 2001, 13 tasks developed by School J had been included in the PT database, and 
another 15 were in their final stages of review and likely to be accepted. By the end of the third CSRD year, 
the developer believed that the school’s staff had still not received training on several key concepts related to 
PT, and without additional funds might not receive this training. 


6. Reorganization of the School Day or School Operations (Component 1). No such changes 
accompanied the reform efforts at School J. 


7. Parental and Community Involvement (Component 6). Parental involvement has proven difficult, 
especially because parents previously at home have been required to return to work as part of welfare reform. 
Parents not speaking English well also have been reluctant to become involved in school activities. At the 
same time, the home-school coordinator stated that she saw a steady increase in the number of parents 
volunteering in classrooms, serving on field trips, assisting teachers, and assisting with afterschool and 
summer programs. Logs show the participation of over 200 parents during 2000-01, with a high number sign- 
ups for workshops for parents during the same year. However, it is notable that no parents were members of 
the core steering committee. The school improvement plan requires a school improvement team, which 
includes two parents. Logs of the parent advisory group show attendance from 15 to 25 members. 


Local businesses and community agencies have been vital in providing support to the school, including 
transportation, mentors, teaching interns, and community roundtables. The school also received support from 
one businessman who solicited a donation of $25,000 for new computers, donated a laser printer, also taught 
a junior achievement class at the school, and is urging local government to provide additional resources, such 
as a playground for School J. 


8. Implementation (Component 1). In the first CSRD year, about half of School J’s teachers were 
trained to use PT. The following year, all but three of the teachers were trained. By the third year, PT was 
being implemented to some extent in all classrooms. However, teachers are given considerable discretion on 
how and when to use the method, and assessing the true extent of implementation is difficult. Further, 
mathematics teachers believe there is an insufficient number of performance tasks related to that subject to 
make the method useful. Two new teachers stated that the complexity of the method made it overwhelming 
for them as beginning teachers. At the same time, classroom observations showed that the language of the 
method was evident in both teachers’ and students’ conversations and revealed other numerous indicators of 
PT in place. According to the external consultant, fidelity appears to medium; she estimated it would take 
five years for staff to fully implement the method with full fidelity. 


9. Professional Development (Component 3). Professional development was provided both by the 
developer and the district staff. In the second CSRD year, two and one-half days were devoted to PT training. 
In the first two years of CSRD, most of the district’s professional development did not appear to be tailored to 
school reform, although by the third year and with the IFL program having been adopted, the focus increased. 
Some training in the third year was conducted after hours. Teacher responses to the quality and organization 
of the professional development has been mixed, with the afterschool requirements considered demanding but 
with other teachers reporting greatly increased use of the computer or comfort with it. In addition, those most 
experienced in the method have served as mentors, or “scouts” to the other teachers. 


10. Other Forms of Support for the Staff (Component 5). No other forms of support were identified. 
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11. Measurable Goals and Benchmarks (Component 4). Classroom implementation of PT has been 
appraised through classroom visitations by members of the school’s core committee. Performance goals for 
the school have been presented through 2001-2002 in the school improvement plan and the CSRD progress 
report. As an illustrative benchmark, reading goals at all grade levels and on all tests were to improve by 10 
percent. The PT also emphasizes assessment as an integral part of school improvement efforts, so that 
benchmarks for reform are assessed on an annual basis. 


12. District Influences (Other External Conditions). The district has supported School J’s reform 
efforts, especially since 2000-01, when the new superintendent took office. The support also derives from the 
superintendent’s concomitant adoption of the IFL professional development model, districtwide, as well as 
the making of balanced literacy a priority in all schools. 


13. State Influences (Other External Conditions). The state’s main role has been in the assessment 
area, where a new assessment was put into place in 2000-01. However, the results will not be used to identify 
schools “in need” until the fall 2003. In the meanwhile, the school is one of the state’s 28 “priority schools,” 
and the state has assigned a “critical friend” to the school as well as extra funds for additional staff support. 


14. Evaluation (Component 8). The school has had an external evaluator who also has provided 
training on understanding state standards, performance measures, portfolios, portfolio assessments, and 
mastery tests. The district evaluation person has assisted the school staff in interpreting state assessment data, 
and test results have been used to identify children for summer school, Saturday academies, and tutoring. 
Schools are required to discuss test results with parents two times each year. 


15. Convergence of Resources and Sustainability (Component 9). Multiple external sources were used 
to support PT, but after CSRD ends, the sustainability of the reform is in question. First, Title I funds may be 
used in the future. Second, the associate principal is being considered for the principalship and, if selected, 
his support for PT is likely to help the school sustain its reform efforts. He readily admitted, however, that 
only some components of the PT method may endure. 
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School K 


Characteristics 


Type of School: Elementary (PreK — 6) 
Population: 613 students 
Ethnicity: 82% minority 


Poverty Rate: NA 
Teachers: 32 (FTEs) 
Principal Turnover: Fall 2000 
Superintendent Turnover: Summer 2000 


1. Research-based Method (Component 1). School K was designated by the state as a school in need 
of improvement in March 1998. Such schools are required by the state to select a proven method in their 
effort to meet state performance standards. This led the district to exposing the school’s staff to SFA and 
submitting a CSRD application in the fall of 1998. School K adopted SFA (roots for grades K-1, and wings 
for grades 2-6) in 1999-2000. No other models are in use. The staff also decided not to start math component 
of SFA until reading was fully implemented. 


2. Staff Support (Component 5). At School K, 80 percent of the faculty and support personnel voted 
(by show of hands) to implement SFA. The district also may have encouraged the adoption, as School K 
appears not to have had any other options. 


3. Leadership Role of the Principal (Component 5). A new principal started in 2000-01 and 
enthusiastically supports SFA, even though it had started under the previous principal. 


4. Comprehensiveness (Component 2). The school improvement plan, also considered a school reform 
plan, addresses all of the CSRD components. A steering committee manages the implementation and 
integration of reform components. Classroom observations and discussions with teachers appear to confirm 
that the staff understand the breadth of the reform strategy. 


5. Support from an External Developer (Component 7). The developer has provided extensive training 
and technical assistance, starting with the training in August 1999 and also including the Family Support 
Team (see item 7). 


6. Reorganization of the School Day or School Operations (Component 1). SFA requires a 90-minute 
block and also the regrouping of students every 8 weeks (however, students in grades 5 and 6 do not get 
regrouped to primary grades). School K also has another 90-minute block for English/language arts, 
following the district curriculum but using many SFA techniques. School K started as a K-3 school in 1999- 
2000. During 2000-01 it was renovated and expanded, and by 2001-02 it was a K-6 school. 


7. Parental and Community Involvement (Component 6). A home-school coordinator offers parent 
training, and the school strongly encourages home reading. Training sessions are flexible and multiple, and 
parents are encouraged to bring children to PTA meetings, where meals are served. About 100 parents are 
available for once-a-month volunteering of some kind. Despite records, there is no clear evidence of change 
in the levels of parent involvement. A Family Support Team carries out the functions of the former Student 
Assistance Team, using case management techniques for individual students. 


D-27 


8. Implementation (Component 1). SFA is in place each day with all students, deserving a ranking of 4 
out of a 5-point scale. School K’s Title I reading specialist became the full-time program facilitator for SFA, 
in addition to other paraprofessionals who provide tutoring services to students. 


9. Professional Development (Component 3). Half of the faculty participated in a five-day workshop 
in August 1999. Trainers made 23 person-visits during 1999-2000, and new teachers also received training. 
Training was repeated in August 2000, and all had received refreshers by May 2001. Teachers also 
participated in district training, but offerings did not necessarily focus on reform. The district also provided 
training on using school profile data and preparing for the state assessment. 


10. Other Forms of Support for the Staff (Component 5). No other forms of support were identified. 


11. Measurable Goals and Benchmarks (Component 4). The school assessed progress toward reform 
goals every year, reporting progress and revisions in the school improvement plan and CSRD end-of-year 
reports. Progress toward goals in reading, problem solving, and writing were measured by student 
performance on state assessments and SFA-developed assessments. School K’s students showed gains in 
2000-01 compared to the previous year, on the Stanford 9 assessment, also equaling or exceeding the district 
average in all grades except grade 5. However, its grades 4-6 only started in 2001-02. 


12. District Influences (Other External Conditions). The district sends each school the results of state 
and district assessments. 


13. State Influences (Other External Conditions). The state played a significant role in the reform 
effort at School K by identifying the school as in need of improvement. The state provided the school with an 
accountability grant, and required the school to select a proven reform method and apply for a CSRD grant. 


14. Evaluation (Component 8). For grades 4-6, teachers and students have complained about the 
rigidity of SFA and voiced pleasure at cancellations of the 90-minute reading period. SFA trainers believe 
that the needed flexibility does exist, but there has been no follow-up training to address this issue, and it is 
still unresolved. 


15. Convergence of Resources and Sustainability (Component 9). Title I funds are expected to be used 


to continue SFA support after CSRD funding ends. However continuation also depends on improved student 
performance as well as SFA’s response to teachers’ concerns. 
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School L 


Characteristics 


Type of School: Elementary School (K — 6) 
Population: 502 students 
Ethnicity: 82% minority 


Poverty Rate: 92% free or reduced lunch 
Teachers: 20 (FTEs) 
Principal Turnover: Summer 1999, Summer 2001 
Superintendent Turnover: None 


1. Research-based Method (Component 1). School L chose to implement a locally-developed method, 
the Behavioral Modification (BEH) method. The method consists of a behavioral protocol, climate-setting 
strategies, and instructional approaches: cognitive-behavioral instruction, individual and class evaluation of 
daily and weekly progress in reading and mathematics, and classroom management using the “Classroom 
Behavioral Game. Students are organized into teams, given jobs, walk in “model lines” and use “model 
voices,” and are asked to monitor classroom behavior. Other elements of the program include having 
students graph their own academic performance. In addition to this CSRD method, School L adopted Open 
Court for reading and SAXON for mathematics, based on the district’s preferences. Because BEH was not a 
curriculum, the method was highly compatible with both of these other methods. In addition, School L 
implemented a violence prevention programs called Second Step. 


2. Staff Support (Component 5). The teachers on the school leadership team voted to adopt BEH in the 
summer of 1999, but no vote was taken with the rest of the staff. The principal reported that everyone 
supported the decision to create order in the classroom so that instruction could take place. A new principal 
was assigned for 1999-2000, when BEH was first implemented, and this principal also was very supportive of 
the method. 


3. Leadership Role of the Principal (Component 5). The principal was the main force for coordinating 
and leading reform. The principal provided incentives for teachers—e.g., teachers successful in getting five 
parents to back-to-school night received $100 in supplies. The principal also encouraged model teaching and 
participation in professional development. This principal then retired after 2000-01, and the new principal 
was not convinced that students believed in the method, so support for BEH practices may have started to 
wane. 


4, Comprehensiveness (Component 2). Reform at School L included attention to all nine CSRD 
components. The plan for reform was part of a district-mandated plan, with a set of determinants of progress 
called “vital signs”—absolute (not relative) benchmarks for student performance. The plan then identified the 
activities to be undertaken to attain these vital signs. The plan also identified the person leading the activity, 
the start and end dates, and the source of funds. 


5. Support from an External Developer (Component 7). Both the district and school report that the 
method developer has provided consistent and strong assistance since the inception of the program. The 
developer helped School L select its CSRD model and assisted this school as well as others in its CSRD 
applications. After the award, the developer acted as a technical assistant and training provider, also visiting 
classrooms three to four times each year and serving as a facilitator at the staff retreats. In addition, the 
developer invited the principals of the schools implementing BEH to a catered dinner at her home a few times 
each year. 
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6. Reorganization of the School Day or School Operations (Component 1). BEH directly affecting 
classroom operations. In addition, the school started a site council. However, no other changes in school 
operations were noted. 


7. Parental and Community Involvement (Component 6). Report cards were not mailed home, so 
parents and teachers met face-to-face, at school, the home, or a local restaurant. About 90 percent of the 
parents attended these conferences. The school also sponsored a variety of other events to attract parent 
attendance, but few parents were involved in classroom activities and parents on the school leadership team 
were often absent. Despite all these efforts, the method developer stated that the “school community 
(including parents) is disengaged from the school.” 


8. Implementation (Component 1). School L started implementing BEH in October 1999, with a 
phased plan. By spring 2001, the principal indicated that about half of the teachers were implementing the 
method. With one exception, most of the classroom observations revealed that teachers were implementing 
BEH. Comments from the staff indicated that about 70 percent of the staff were using the method as 
intended, but that there were some staff members who could benefit from additional training. By spring 
2002, implementation was beginning to falter, perhaps because of a change in school leadership. Further, 
there was little evidence that instructional components were understood by staff or used in the classroom. 
Overall, implementation progress was judged as a “3,” or still in the piloting stage. The method developer 
reported that School L was implementing the method with fidelity, when implementation occurred. 


9. Professional Development (Component 3). All staff members were required to attend three days of 
inservice training on BEH in August 1999, followed by retreats in May 2000 and Spring 2001. CSRD funds 
paid for the expenses. At least four of the professional development events, involving a total of nine days in 
1999-2000, were related to BEH. The district also required 18 hours of professional development annually, 
providing assistance on both the reading and mathematics curricula used by the school. All teachers 
interviewed said they believed the professional development was beneficial and sufficient to meet their needs 
in implementing reform. 


10. Other Forms of Support for the Staff (Component 5). Teachers also met on a bi-weekly basis, once 
to discuss staff issues and once to discuss curriculum topics. Teachers recorded 60 grade level meetings 
during 2000-01. The school also had a resource teacher who helped facilitate meetings of the leadership 
team—a school site council that had, as part of its responsibilities, assistance in implementing and monitoring 
the school improvement program. 


11. Measurable Goals and Benchmarks (Component 4). The district adopted a strategic plan in 1998 
that set school goals for 2001. These goals pertained to student performance and were cast in absolute terms 
(e.g., 95 percent student attendance), applied to all schools even though they might have been at different 
starting points, and were posted in classrooms and in public hallways. Other benchmarks for implementing 
BEH were stated in the CSRD application and were monitored by the external developer. According to 
teachers and administrators, the most direct evidence of the success of the reform effort was the greatly 
improved behavior of the students. In 1998-99, the school had the highest suspension rate of any elementary 
school in the district, and attendance was only 93-94 percent. Since that time, the number of suspensions 
have dropped substantially and attendance has increased by 2 percentage points. 


12. District Influences (Other External Conditions). The district played a major role in selecting the 
CSRD schools, and its strategic plan adopted in February 1998 under a new superintendent has been a driving 
force behind School L’s reform efforts. The district also provided monetary incentives to teachers were 
students’ test scores improved, and for performance during 1999-2000, each teacher at School L received 
$570. 


13. State Influences (Other External Conditions). The basic driving force behind reform at the school 
and district was the need to improve performance on the state assessment. However, the state was in the 
process of revising its assessments to make them standards-based, leading schools to refine their curricula 
further. 
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14. Evaluation (Component 8). The contract with the external developer also called for an external 
evaluation by another staff person. However, the evaluator made few observations during 2000-01, and by 
fall 2001 no report had been produced. The district annually prepared color-coded charts depicting state 
assessment results, although the data analysis was not entirely consistent with district goals (e.g., defining 
“below grade level” as scoring in the 0-24 percentile on the SAT-9). 


15. Convergence of Resources and Sustainability (Component 9). The district planning process 
brought all dollars into a single reform effort. A possible threat to sustainability was the principal’s 
retirement. Funds to replace CSRD were still being sought, as BEH is not a scripted instructional method and 
requires a certain level of professional development and monitoring to be sustained. 
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School M 


Characteristics 


Type of School: Elementary (K — 8) 
Population: 680 students 
Ethnicity: 97% minority 


Poverty Rate: 87% free or reduced lunch 
Teachers: 55 (FTEs) 
Principal Turnover: Spring 1998, Spring 1999, 
Summer 1999, Fall 1999 
Superintendent Turnover: None 


1. Research-based Method (Component 1). School M used CSRD to adopt Lightspan (LT), starting 
with its 3rd grade in 1999-2000 and then in its 3-5th grades in 2000-01. In this research-based strategy, 
parents and students sign contracts to vouch for home-use of a PlayStation computer and to borrow software 
from the school. The school also has PlayStations in the classrooms, so that lessons are taught at school and 
the home PlayStations can be used for homework exercises. The PlayStations are returned to the school at the 
end of the school year. After the original principal resigned, however, LT was not being used in classrooms 
or at home during 2001-02. 


2. Staff Support (Component 5). The original principal recruited staff who were supportive of LT. 
However, with his departure staff support for LT has diminished. The new principal notes the lack of data on 
increased parent involvement, which might have provided some support for continuing the method. 


3. Leadership Role of the Principal (Component 5). The original principal took over in 1997-98, hired 
over 80 percent of the current staff, and was highly supportive of LT. He believed that parental involvement 
with students at home was the key difference between high- and low-performing students. He guided the 
implementation process through the first two CSRD years (1999-2000 and 2000-2001), but by the spring of 
the second year, he abruptly resigned from the school. Since then, the school has had an interim principal 
(April-August 2001) and a permanent replacement (August 2001-present). The new principal and staff do not 
appear to have LT as one of their priorities. 


4. Comprehensiveness (Component 2). The school’s original three-year site plan included LT but did 
not cover reform components more broadly. Starting in August 2000, the district revised the school 
improvement plan and its process, so that the plan is more comprehensive and more aligned with state and 
district standards. The school only completed its initial revised plan in June 2001, so that the new process and 
provisions have not yet had a chance to have clear effects. 


5. Support from an External Developer (Component 7). The developer (TA provider) has provided 
extensive on-site support. During the first year, the support was not as consistent as in the second year, when 
a new person served as the TA provider. The TA provider rated School M’s use of Lightspan as among one 
of the highest among the more than two dozen schools he had worked with, especially in the high rates of 
home deployment. 


6. Reorganization of the School Day or School Operations (Component 1). Changes in the internal 
organizational structure were not encountered during any of the site visits. During 2001-02, and as a result of 
state budget cutbacks, School M’s busing pattern and school day shifted substantially, so that students were 
only starting their school day at 9:30 a.m. 
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7. Parental and Community Involvement (Component 6). Parents had to sign a contract for home 
deployment, and about 90 families did so in 2000-01. However, after the original principal resigned, 
deployment has been discontinued, and parent interest in the annual school events associated with LT has 
diminished. 


8. Implementation (Component 1). Even during 2000-01, when LT was supposed to be fully deployed 
in three grades, classroom observations showed that implementation was not occurring in several of the 
classrooms. 


9. Professional Development (Component 3). All grade 3-5 teachers and some staff received LT 
training during 2000-2001. Additional on-site training has been provided by the LT developer. Teachers will 
need additional training if LT is to be continued during 2001-2002. The district provides 3-4 additional 
professional development days, but there has been no coordination to assure that these days are related either 
to LT or to reform more generally. 


10. Other Forms of Support for the Staff (Component 5). The school has some control over its whole 
budget and can direct resources to professional development. However, the professional development 
priorities of the new principal are unknown. 


11. Measurable Goals and Benchmarks (Component 4). The district collects a broad variety of student 
performance data and makes these data available to the school. These data have mainly been used as part of 
the self-evaluation of LT described below. 


12. District Influences (Other External Conditions). The district gave School M a Quality 
Performance Award for its performance during 1999-2000, reflecting better than average gains in its 
accountability system. For the same year, however, the state placed School M in its lowest underperforming 
category and started providing the school with compensatory funds. There is no clear explanation for the 
discrepancy in the two ratings, especially since the district and state assessments are said to be correlated. 
School M also was transitioning from a K-6 to a K-7 and then a K-8 school. Further, due to the poor 
performance of a nearby K-8 school on the state assessments (in fact, the nearby school shared the same 
school building as School M), the schools will merge in some fashion starting in 2002-03. The pattern of 
staff retention and overlap were unknown. 


13. State Influences (Other External Conditions). The state’s placement of School M in its lowest 
underperforming category reduced the morale and esteem of the school and its staff. 


14. Evaluation (Component 8). The school was conducting its own self-evaluation of LT. Data were 
collected and results analyzed by May 2001, but this was too late to become part of the revised school 
improvement plan. The evaluation findings did suggest that LT students outperformed comparison students 
in the state assessment. 


15. Convergence of Resources and Sustainability (Component 9). The original principal appears to 


have converged resources around LT, its coordinator, computer equipment, and the other resources needed for 
implementation. However, the present principal does not appear at all committed to LT or its continuation. 
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School N 


Characteristics 


Type of School: Elementary (PreK — 5) 
Population: 510 students 


Ethnicity: 98% minority 
Poverty Rate: 28% free or reduced lunch 
Teachers: 30 (FTEs) 
Principal Turnover: None 
Superintendent Turnover: Winter 1999, Winter 2000 


1. Research-based Method (Component 1). School N voted to adopt Success for All (SFA) during the 
1997-98 school year, when every teacher spent time observing classrooms at schools already implementing 
SFA and 98 percent of the faculty voted to adopt SFA. In addition to SFA, School N also used Accelerated 
Reader as a supplement to SFA and a reading program focused specifically on skills addressed by the state 
assessment. School N also had been using Everyday Mathematics until 1999-00, when it started using a local 
program for all schools, based on a specific textbook that the district believed was better aligned with the state 
assessment. 


2. Staff Support (Component 5). The faculty was strongly supportive of SFA at the outset. The model 
was one of several New American Schools’ models that was strongly encouraged by the district 
superintendent, and some teachers commented that SFA was the best of the “choices that were given.” 
However, once the faculty decided to implement the program, they “wanted to do it right.” Although the 
school had some carryover funds, 2001-2002 was the first year of implementation after the CSRD award had 
ended, and teachers expressed mixed reactions about whether they would favor continuing the program 
during 2002-03. Teachers also expressed concern that they did not have many of their own students during 
the 90-minute reading block and yet the district was holding teachers accountable for ensuring that their 
students could read at grade level. Finally, in 1999-00 the principal was forced to discontinue the practice of 
assigning the non-traditional classroom teachers due to labor union issues, losing six teachers who had been 
working during the block and leading to reduction of teachers’ support for SFA. 


3. Leadership Role of the Principal (Component 5). According to staff and faculty, the principal and 
her administrative assistant were important participants in the implementation process. The principal also had 
led the school in making its initial adoption decision, later adapted the school’s schedule and staffing patterns 
to meet the needs of the reform effort, and defined other resources needed to support the effort. 


4. Comprehensiveness (Component 2). All 9 elements of comprehensive reform were present in the 
school. In 2001, the district designed a new format for the school improvement plan, tied directly to the state 
assessment and used in 2001-02. The plan sets goals, but the strategies for reaching the goals can be stated in 
a general manner. 


5. Support from an External Developer (Component 7). SFA facilitators provided extensive training 
and conducted site visits twice a year through 2000-01 and once in 2001-02. During these visits, observations 
and interviews were conducted, followed by a discussion of strengths and areas of need. This process was 
conducted in a supportive, nurturing manner, and the more experienced SFA teachers indicated that the 
assistance was beneficial because it brought an objective perspective to the work they were conducting. 
Although the support was of high quality, as external support receded during 2001-02, questions regarding the 
level of training available for new teachers began to arise. 


6. Reorganization of the School Day or School Operations (Component 1). Starting in 1999-00, when 
SFA was implemented throughout the school, the principal rearranged the schedule to allow for a schoolwide 


D-34 


90-minute block. The principal also arranged instruction periods for other subjects to correspond by grade 
level, in turn allowing for corresponding planning time by grade level. In addition, the librarian, resource 
teacher, physical education instructor, music teacher, and other teachers all were trained in the SFA 
curriculum, allowing a slightly reduced average class-size during the reading block. 


7. Parental and Community Involvement (Component 6). Overall, despite significant efforts, parents 
interviewed in the spring of 2002 indicated that parental involvement had declined during the past few years. 
The school had developed a variety of programs to increase parental involvement—focused directly on the 
reform efforts at the school—but in 2001-02 the school had to drop its parent coordinator to save money. 
Support of school activities by business or community groups was minimal and did not change much during 
the reform period. 


8. Implementation (Component 1). Implementation began in 1998-99. Full implementation, affecting 
the entire school, did not occur until a year later. CSRD support then continued for a third year, 2000-01. By 
that time, all classrooms appeared to be using SFA during the designated time, although some teachers had to 
cut off important and thoughtful discussions in order to remain on the strict time schedule set by SFA. Until 
2000-01, reading and mathematics were the main focus of implementation; district-mandated quarterly- 
assessments led the school to placing greater emphasis on science and social studies during 2001-02. 


9. Professional Development (Component 3). Prior to 1997-98, PD was coordinated centrally. 
Starting in 1998-99, the PD model was modified to a trainer-of trainers model so that PD would increasingly 
take place at the individual schools. Teachers received a considerable amount of PD support for SFA. 
However, the priorities for PD in general went beyond any single method and reflected a combination of 
district- and school-level goals for improving teaching and learning. District offerings related to 
comprehensive reform and the contents of the School Improvement Plan. Although PD requirements 
increased during this time (to 12 hours?), some professional development was to be completed on personal 
time, without stipends. 


10. Other Forms of Support for the Staff (Component 5). The grade-level planning period appears to 
be an important source of support for the staff. 


11. Measurable Goals and Benchmarks (Component 4). The main benchmarks have been contained in 
the school improvement plan. However, the plan sets goals and objective for a 5-year period but only 
contains timelines for implementation for the immediate school year. For the school, the principal stated that 
the goal was to attain the “exemplary” (highest) level of achievement in the state’s ranking system. For 2000- 
01, the school achieved a “recognized” (second highest) status. 


12. District Influences (Other External Conditions). The district played a major role in the CSRD 
implementation. First, the original district superintendent required all schools adopt one of the models from 
the New American Schools initiative. Second, a later superintendent who assumed office in 1999, decided not 
to support these models, instead favoring a district-developed curriculum that was assumed to be better 
aligned with the state assessment. Third, in August 2001, the district released new content standards that 
provided teachers with a detailed guide to the skills that students were expected to learn each week. Fourth, 
the district started high-stakes accountability, holding teachers responsible for their students reading at grade 
level, which was not compatible with the cross-age grouping promoted by SFA. This accountability further 
diluted support for SFA because teachers did not have sufficient time with their own students during SFA’s 
90-minute blocks. The same district-mandated quarterly assessments led the school to place greater emphasis 
on science and social studies in 2001-02. 


13. State Influences (Other External Conditions). The state also has had a high-stakes assessment in 
place for some time. This has led to alignment of district curricula, the school improvement plans, and 
professional development practices. For 2000-01, the school achieved “recognized” status (the second 
highest ranking) in the state assessment system. Overall, district and state policies have driven School N’s 
reform more than has CSRD. 
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14. Evaluation (Component 8). There has been no evaluation of the process of reform except for the 
implementation reports completed by SFA. 


15. Convergence of Resources and Sustainability (Component 9). To implement the instructional 
changes identified in the SIP, the district and school coordinated funding, although they relied exclusively on 
Title and CSRD. Most of the funds received by the school from the district office had already been 
budgeted for specific activities. The principal was skeptical about whether the school would continue to 
implement CSRD without any new funding, and the staff indicated that the school would not be implementing 
SFA in 2002-03 even if funding were available. 
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School O 


Characteristics 


Type of School: Elementary (PreK — 5) 
Population: 476 students 
Ethnicity: 96% minority 


Poverty Rate: 90% free or reduced lunch 
Teachers: 34 (FTEs) 
Principal Turnover: None 
Superintendent Turnover: Winter 1999, Winter 2000 


1. Research-based Method (Component 1). School O, a pre-K-5 school, started using SIM in grades 
1-4 during 1996-97—prior to the CSRD award. The award permitted the school to expand its use of SIM to 
kindergarten and the 5th grade. This pattern is different from the school’s original CSRD application, which 
called for adopting Co-NECT; but after the school received a low ranking in the state’s accountability system 
in 1996-97, the district encouraged the school not to use CSRD to start a new research-based method but to 
expand its then-current one. In 2000-01, the school also began transitioning away from SFA, which had been 
adopted in 1996-97, to the district’s comprehensive reading curriculum. 


2. Staff Support (Component 5). At School O, 93 percent of the faculty voted in a secret ballot to 
expand the use of SIM. At the same time, the faculty also had previously voted to adopt Co-NECT, and to 
coincide with the district directive, also had voted in 2000-01 to abandon its use of SFA. 


3. Leadership Role of the Principal (Component 5). The principal created a leadership “cabinet,” 
consisting of the lead teachers in math, science, and reading, the coordinator for the magnet project, the SIM 
coordinator and media specialist, the counselor, and the assistant principal. The cabinet meets with the 
principal weekly to plan and review progress on all school reform efforts, including SIM. 


4. Comprehensiveness (Component 2). The school improvement plan refers to a number of reform 
components, mainly the use of the research-based method, aligned professional development, and parent 
involvement. The plan is produced through deliberations by a school advisory council. The breadth of the 
planning process and SIM appear to provide a unifying plan of action throughout the school. 


5. Support from an External Developer (Component 7). The developer has provided constant and 
welcomed technical assistance since the school started using SIM in 1997. 


6. Reorganization of the School Day or School Operations (Component 1). Teachers and students 
were required to adjust their daily schedules to allow for 40 minutes of computer instruction (20 for reading 
and 20 for math) per day. Some teachers attempted to accommodate this change by continuously rotating 
students through the software program throughout the entire school day. Students had to learn to work with 
their peers in order to catch-up to the teacher-directed instruction upon finishing the computer work. 


7. Parental and Community Involvement (Component 6). Obtaining parent involvement has been a 
constant challenge. Babysitters are provided to parents during PTA and school improvement team meetings, 
and meals also may be served. 


8. Implementation (Component 1). Implementation scale-up has been completed. Teachers are 


required to use SIM math software for a minimum of 20 minutes a day and also SIM reading software for 
another 20 minutes. SIM developers recommend that students spend 25-35 hours of computer time per year 
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to make greater achievement gains, and teachers have had to incorporate computer time into their classroom 
routines as well. In 2000-01, the staff voted to drop the mandatory daily use of the software, instead allowing 
teachers to integrate the software into their lessons plans more flexibly. Classroom observations have shown 
varied implementation patterns, with some classrooms not using SIM at all. SIM use appears especially to 
decline during the weeks prior to state tests. 


9. Professional Development (Component 3). Professional development has emphasized the district’s 
comprehensive reading program. New staff also received SIM training, and existing staff received some 
refresher training. The SIM developers also provide a lot of on-site training. 


10. Other Forms of Support for the Staff (Component 5). Staff receive stipends to attend Saturday 
trainings on curriculum goals, instructional strategies and magnet school topics. 


11. Measurable Goals and Benchmarks (Component 4). The SIM software includes collecting data on 
the progress being made by individual students, but these data were not being reviewed as frequently by 
teachers in 2001-02 as in the past. District-mandated benchmark testing (every nine weeks) as part of the 
comprehensive reading program are possibly receiving greater priority. 


12. District Influences (Other External Conditions). The district has emphasized student achievement 
and school improvement efforts. Once assigned to schools, principals can choose to maintain or replace any 
element of a school’s instructional program, however. CSRD may be being used as another program and not 
as a catalyst for reform. 


13. State Influences (Other External Conditions). The state has started its own accountability system. 
Since School O’s initial low rating in 1996-97, its state rating had improved sufficiently that, by the fall of 
2001, it was no longer in the “jeopardized” category. In addition, from 1999-2000 to 2000-01, significantly 
improved performance on writing tests has been reported. 


14. Evaluation (Component 8). The district will be conducting an impact assessment during 2001- 
2002, as part of its broader plan to evaluate all CSRD research methods. No other evaluation activities in 
relation to CSRD appear to have been in place at School O. 


15. Convergence of Resources and Sustainability (Component 9). The school has successfully 


coordinated its available federal resources and directed them coherently toward reform activities. The extent 
of integration with the internal school funds is unknown. 
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School P 


Characteristics 


Type of School: High School (9 — 12) 
Population: 645 students 
Ethnicity: 67% minority 


Poverty Rate: 72% free or reduced lunch 
Teachers: 39 (FTEs) 
Principal Turnover: Summer 2000 
Superintendent Turnover: Summer 2001 


1. Research-based Method (Component 1). School P adopted High Schools That Work (HSTW), a 
comprehensive method that combines traditional academic content with vocational courses and that promotes 
9 key practices. A state-funded Tech Prep grant was a complementary method aligned with HSTW and 
considered a component of HSTW’s professional development plan. Likewise, INTECH was a small state 
grant providing hardware, software, and teacher training to integrate technology into the classroom, one of 
HSTW’s strategies. 


2. Staff Support (Component 5). The staff had no input into the original principal’s decision to adopt 
HSTW. However, a second principal worked hard to engage the staff in the reform efforts, and a year later 
most faculty were supportive. For instance, the faculty voted nearly unanimously to shift to a 4X4 block 
schedule, starting in the fall of 2001. 


3. Leadership Role of the Principal (Component 5). The initial principal was reform-minded and 
research-driven and identified HSTW prior to the CSRD grant. The principal then oversaw HSTW’s initial 
implementation year (1999-2000). However, the principal resigned in 2000. A new principal, hired in late 
August 2000, learned the reform process and made a concerted effort to include everyone in the adoption and 
implementation efforts. This principal then left the school at the end of 2001-02. 


4, Comprehensiveness (Component 2). The implementation of HSTW was accompanied by a 
comprehensive design covering all of the CSRD components. By exposing teacher leaders to extensive 
HSTW training and creating opportunities for sharing this training with other teachers, the 
comprehensiveness of the plan has become well understood and also implemented throughout the school. 


5. Support from an External Developer (Component 7). The district has contracted with the HSTW 
developer for a variety of annual services during the CSRD award period, including on-site and technical 
review visits, monthly telephone conferences, and an HSTW assessment. Some of these activities also should 
continue after the third CSRD year, as the school has budgeted $10,000 for 2002-03 to cover the developer’s 
services. These funds come from a grant for Making Schools Work, which started in the final CSRD year and 
is another method developed by the HSTW developer. 


6. Reorganization of the School Day or School Operations (Component 1). Besides adopting a 4X4 
block schedule in 2001-02, School P also installed a teacher advisement program (Wednesday club) whereby 
students met at least every two week with the same teacher throughout high school, developing a plan tailored 
to the student’s interests and strengths. In addition, HSTW also involves an active role by a school 
leadership team. 


7. Parental and Community Involvement (Component 6). Parent involvement has been difficult to 
achieve at School P. For instance, since implementing HSTW, the school attempted to form a parent 
advisory group, but without success. Little to no change has occurred in volunteer time or parental 
involvement in reform. 
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8. Implementation (Component 1). Implementation of HSTW took place over a three-year period 
(1999-2000 to 2001-2002). Much of the initial year was spent on needed professional development, and only 
by the third year did the school implement the 4X4 block schedule recommended by HSTW. By the fall of 
2001, the site visit team gave School P the highest implementation rating (a “5”) on the Bodilly scale. The 
HSTW representative indicated that, based on her observations, School P’s fidelity to the original method 
was “high.” 


9. Professional Development (Component 3). According to HSTW leadership team members, the 
structure of professional development has changed drastically at School P. Prior to HSTW, the activities 
were uncoordinated and did not follow an identified strategy. Since CSRD, the activities have become well- 
orchestrated and directed, with all of the CSRD funds being devoted to professional development, including 
stipends for the teachers, fees for the external developer, teacher replacement costs, and travel and conference 
fees. During 1999-2000, the faculty averaged 8.1 staff development days, compared to a district average of 
7.3 days. About 75 percent of the teachers have taken more than the required professional development, with 
teachers teaming with teachers being a primary HSTW strategy. In CSRD’s third year, School P’s staff also 
received additional professional development in relation to helping students transition from middle to high 
school. 


10. Other Forms of Support for the Staff (Component 5). No other forms of support were identified. 


11. Measurable Goals and Benchmarks (Component 4). The school has a formal benchmark document 
that is continually monitored and updated and that includes comprehensive, schoolwide indicators of quality 
(e.g., reading comprehensiveness) that are monitored by the school and can become the basis for remedial 
action by the faculty. The benchmarks include both outcome measures and organizational and curriculum 
strategies. 


12. District Influences (Other External Conditions). The district provided additional funds for 
professional development related to HSTW. A potential negative factor has been political and social conflict 
involving School P, and the superintendent also resigned precipitously at the end of 2000-01. In November 
2001, voters approved a tax referendum to renovate existing school buildings, and shortly thereafter, the 
board voted to merge School P with the district’s other high school, for 2003-04. How teaching staffs are to 
be merged, and with what support for reform, has yet to be decided. 


13. State Influences (Other External Conditions). The state has developed an end-of-course 
assessment that will eventually replace the state graduation test requirement. The state has also increased the 
academic graduation requirements for students completing vocational high school programs. School P added 
a math course (money management) to provide additional opportunities for its vocational students to meet 
these requirements. The state is also conducting its own evaluation of CSRD (see next item). 


14, Evaluation (Component 8). The leadership team and the HSTW’s procedures provide ongoing and 
formative evaluation. including the review of student achievement data and the results of school climate and 
teacher surveys. In addition, the state is conducting its own evaluation through a contract with two research 
firms. The evaluation calls for collecting data from a variety of sources, including classroom observations. 


15. Convergence of Resources and Sustainability (Component 9). The school coordinates use of 
federal and state funds, and the CSRD on-site coordinator is paid from the school’s core budget. 
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School Q 


Characteristics 


Type of School: Elementary (PreK — 5) 
Population: 407 students 
Ethnicity: 60% minority 


Poverty Rate: 74% free or reduced lunch 
Teachers: 34 (FTEs) 
Principal Turnover: None 
Superintendent Turnover: Summer 2002 


1. Research-based Method (Component 1). School Q has been implementing a district-sponsored 
comprehensive approach to literacy instruction since 1995-96. Starting in 1997-98 (prior to CSRD), the staff 
were interested in implementing a comprehensive method matching the school’s philosophy, and the school 
also later applied for a CSRD award, adopting Accelerated Schools (AS). The CSRD-supported research- 
based strategy sets a framework for whole-school reform by advocating specific school improvement 
processes and practices, but the strategy does not specify any particular curriculum. Included in the strategy 
are: the involvement of entire school staff and collaboration in setting class schedules and curricula, flexible 
student groupings, and also parent involvement. To complement AS, School Q uses a district-sponsored 
comprehensive approach to literacy instruction. The approach is closely aligned with AS. 


2. Staff Support (Component 5). Nearly all the teachers agreed to implement AS as a way to organize 
their reform efforts, to enable students to improve their performance on the state assessments. 


3. Leadership Role of the Principal (Component 5). The school principal has been a strong supporter 
of the research-based strategy and of school reform, has worked to developed a compatible school vision, has 
been in place for several years, and has continually identified funds and other resources to support the reform 
efforts (e.g., professional development monies and time for teacher collaboration—see items 9 and 10). 
Principal also has established leadership team, consisting of cadre leaders and others, to discuss schoolwide 
issues; goal is to have team ensure continuation of reform and avoid reform being dependent on a single key 
person. 


4, Comprehensiveness (Component 2). The comprehensiveness of the reform effort is reflected both in 
the school improvement plan and in the agenda being followed by a reorganization of the school and its 
governance elements (see response to item 6 below). 


5. Support from an External Developer (Component 7). A local university is affiliated with the 
national developer and routinely provides technical assistance to the school’s staff in using AS. Since 1999- 
2000, the center representative has visited the school every six weeks, and the school’s staff has attended 
university-based training three times a year. The Center claims that this level of assistance is greater than that 
provided by the national developer. 


6. Reorganization of the School Day or School Operations (Component 1). The school has changed its 
internal structure, creating teaching cadres, coaches, instructional teams within the classrooms, and an 
integrated standing committee, as recommended by the research-based strategy. All have received 
professional development to know their tasks and provide training to other school staff. 


7. Parental and Community Involvement (Component 6). The grant supports half of a full-time parent 


coordinator, and parents participate in governance and educational activities, but parent involvement still 
remains lower than desired by the research-based strategy. At the same time, parents do participate in school 
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governance and decisionmaking (e.g., serving on the school steering committee), participate in classrooms, 
and are part of formal parent-teacher teams. 


8. Implementation (Component 1). The school has implemented all aspects of AS, including the 
organizing of the cadres. The TA provider believes more attention needs to be given to school climate and 
parent involvement, and implementation is only beginning to affect classroom instruction. Barriers include a 
high rate of turnover among the teachers during the past two years (about 40 percent each year). However, 
some of the turnover may have been because teachers did not want to make the changes associated with the 
research-based strategy. 


9. Professional Development (Component 3). Professional development covers specific instructional 
and pedagogical practices consistent with AS. The professional development includes helping the cadres to 
undertake a detailed analysis of student performance and also focuses on achieving a unity of purpose within 
the school. Despite financial cutbacks, the district also supports three sessions annually to focus on 
districtwide issues such as district testing plans. Overall, much of the professional development is oriented 
toward reform issues and building staff cohesiveness. 


10. Other Forms of Support for the Staff (Component 5). The formation of the cadres, and the time set 
aside for them to work together, represents a form of support for the staff (e.g., weekly meetings with teachers 
of the same grade). The school also has early dismissal days and a two-day retreat in August of each year. 


11. Measurable Goals and Benchmarks (Component 4). The research-based method includes 
monitoring both process and outcome measures, which is followed by the school. School Q also is subject to 
a considerable amount of testing, using both state and district tools. Both have targeted performance levels 
for the school, and because the school was able to convince the district and state to provide test scores 
disaggregated by classroom, the test data also have been useful in suggesting strategies for improving student 
performance. 


12. District Influences (Other External Conditions). The district administers the state assessments and 
its own CRT. The district believes that the state assessments are about two-thirds aligned with district’s 
standards. 


13. State Influences (Other External Conditions). The district budget had been reduced because of cuts 
in state funding. Starting in 1999-2000, the district reduced its support for all professional development and 
evaluation activities (which can still be conducted within the schools’ budgets). The district’s budget may 
again be in deficit for FY2001-02, causing new cutbacks in school budgets. 


14. Evaluation (Component 8). Teachers complain that annual shifts in district’s CRT content disrupt 
cadres’ work, because of potential nonalignment of educational practices and assessment priorities. 


15. Convergence of Resources and Sustainability (Component 9). The school orchestrates Title I, local 
budget, and CSRD funds to support reform activities (each source of funds covers different types of activities, 
but all are complementary). The principal hopes that the formation of a leadership team will make the reform 
efforts less dependent on any single individual and therefore serve as a mechanism for sustaining reform. 
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School R 


Characteristics 


Type of School: — High School (9 — 12) 
Population: 1,526 students 
Ethnicity: | 66% minority 


Poverty Rate: 48% free or reduced lunch 
Teachers: 74 (FTEs) 
Principal Turnover: Summer 1998, Summer 1999, 
Summer 2000 


Superintendent Turnover: None 


1. Research-based Method (Component 1). School R proposed a multi-faceted, comprehensive, and 
customized—hence locally developed method—for CSRD. One component, the use of the state’s graduation 
standards, was described as “substantiated in research” and therefore met the research-based criterion. The 
standards specify a series of “high” standards, representing a “profile of learning,” and students complete a 
standard when they complete the classroom assignment for that standard. School R aligned its curricula with 
these standards, as well as with school-to-work indicators and lifework plans, and these all became key 
components of a method for whole school reform. Other components included a freshman foundation course, 
flexible block scheduling, an interdisciplinary curriculum, and a technology component—the use of 
individually-assigned laptop computers as part of a separate Laptop Computer Project which started with the 
9th grade in 1998-99. 


2. Staff Support (Component 5). Staff support for reform began strong, with possibly 80 members 
serving on six subcommittees during the planning process, but gradually diminished over the years. Some 
teachers—e.g., the mathematics department—appeared not to be involved in the reform activities at all. The 
four-period block schedule has been retained on an annual basis, based on a favorable faculty vote each year. 


3. Leadership Role of the Principal (Component 5). The initial principal who supported the entire 
effort had been in place for four years but departed after the first year of implementation (1997-98). The 
following three years, School R had three different principals. The first two offered no particular support for 
reform, and its activities diminished. As a result, the faculty, with exceptions, had no drive to continue 
pushing for CSRD. The current principal started in 2001-02 and is now in his second year at the school. 


4, Comprehensiveness (Component 2). The breadth of School R’s comprehensive reform, in principle, 
reflected all of the other eight CSRD components. However, the actual documentation for the design did not 
appear in any single place. The school improvement plan, for instance, only lists benchmarks and does not 
identify the interventions or strategies aimed at achieving the benchmarks. Even the original CSRD 
application was short and did not provide a coherent picture. Staff understanding of the design probably 
peaked during the planning years, and since then teacher and staff turnover have further aggravated the 
situation. Currently, rather than being based on any coherent vision, School R’s CSRD-funded activities 
appear to be supported as a series of isolated activities. 


5. Support from an External Developer (Component 7). Because the CSRD research-based method 
was truly locally developed, School R received no external technical assistance after having consulted a 
variety of external parties during the planning period. The laptop project also had no external assistance. 
During 2001-02, School R $15,000 in CSRD funds to support a consultant to help implement the small 
learning communities initiative to be implemented during the following year. 
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6. Reorganization of the School Day or School Operations (Component 1). School R adopted a four- 
period block schedule in its initial year of reform, and this schedule has been retained to this day. The 
schedule allows teachers to have a common (and lengthy) planning period. Also, reform involved the 
creation of a “freshman foundations” course and some interdisciplinary teaching—e.g., reading a Civil War 
book for both social studies and English—but this course was among the discontinued reform activities after 
the initial startup year. 


7. Parental and Community Involvement (Component 6). From interviews, whether parent 
involvement increased or not was unclear. However, from the 1999-2000 student survey, 30 percent of the 
students reported that their family often or frequently came to school for meetings—an increase of 9 percent 
over the previous year. School R also had a number of partnerships with the business community. Firms 
donated funds, and one firm provided nine mentors, across all subjects, who were available three times a 
week. Businesses were accustomed to working with School R, but this was likely to have been a result of its 
magnets, not because of any comprehensive reform model. 


8. Implementation (Component 1). Planning for reform began as early as 1994-95 and continued 
through the next two years under the leadership of a formal redesign committee of 17 persons, with a vision 
covering “total school reform.” During these years, the staff visited other schools, held focus groups with 
parents and students, and received external assistance. Implementation began with the 9th grade in 1997- 
98—a year before the first CSRD year (the CSRD application was only submitted in October 1998). During 
the next three years, however, most of the reform activities were discontinued, in part due to a tumover in 
principals. CSRD funds were underspent, and implementation restarted in 2001-02, focusing on 9th grade 
teams, block scheduling (which had not been dropped), and “my action plan” (MAP) career development 
planning. Because School R totally customized the research-based method, there were no criteria against 
which to assess fidelity. Moreover, in the meanwhile the district had mandated that, for 2001-02, the school 
to plan for a small leaming communities initiative—in which students choose one of four “houses” for their 
entire high school career—to be implemented in 2002-2003. 


9. Professional Development (Component 3). Each teacher completes a professional development 
plan, linking the planned professional development to district and state standards and goals. A three-member 
committee reviews the plan to assure its alignment with the school improvement plan. Because School R’s 
reform was not tied to any of these standards, goals, or plans, there also was no evidence of any link between 
professional development and CSRD reform. The school typically provides 20 hours of professional 
development per year, with the district adding another 3-4 days. 


10. Other Forms of Support for the Staff (Component 5). The MAP curriculum was developed with a 
small group of teachers during the summer of 2001. The teachers received a CSRD-funded stipend for their 
work. 


11. Measurable Goals and Benchmarks (Component 4). The school improvement plan is a three-year 
plan that has explicit benchmarks for five student achievement subjects: reading, writing, mathematics, 
technology, and cooperation. The principal indicated that various entities, such as school counselors, helped 
to track the benchmarks, but the data do not appear to be tracked in a coordinated fashion. The district tracks 
and issues detailed summary reports of school performance, including the results of teacher, parent, and 
student surveys. In 2001-02, School R received a quality performance rating of 2.6 on a scale from 1 (low) to 
5 (high). The score was mid-range and required neither prescriptive measures nor cash awards, which are 
part of the district’s accountability system. 


12. District Influences (Other External Conditions). The district helped School R to develop its CSRD 
application. District school assessment and accountability policies also have been important but have not 
greatly affected School R because of its mid-range ranking. 


13. State Influences (Other External Conditions). The state played an active role, helping to recruit 


School R into CSRD in the first place and then supporting annual meetings of CSRD schools and conducting 
annual monitoring visits to schools. Because of School R’s underuse of its CSRD funds, the state approved a 
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no-cost extension that effectively put the school into the second rather than the first cohort of CSRD awards. 
Cutbacks in state funding had minor effects on the school in 2001-02 but threatened to have a more severe 
impact in 2002-03. 


14. Evaluation (Component 8). An external evaluator had been monitoring progress for a few years, 
but the CSRD method adopted in the fall of 2001 had no evaluation component. 


15. Convergence of Resources and Sustainability (Component 9). A separate reference to CSRD funds 
did not appear in the school budget, and school funds supported the CSRD coordinator role. To this extent, 
some convergence of funds may have occurred. However, the lack of a comprehensive or coherent vision or 
plan appears to make both the issues of convergence and sustainability moot. 
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APPENDIX E 


47-Point Instrument for 
Assessing Strength of CSRD Implementation at 18 Schools 


Appendix E 


47-Point Instrument for 
Assessing Strength of CSRD Implementation at 18 Schools 


Component Measure Score* 
1. Research-Based Method or Strategy 
1.1 Implementation Score (adjusted Bodilly Scale): 
1.2 Percentage of classrooms using that should have been using: 


1.3 Fidelity rating by developer or consultant (high, medium, low, defined as 
follows: 


high: developer/consultant considers school to be among the best seen 

medium: developer/consultant considers school to be using method in 
acceptable manner 

low: developer/consultant has major complaints about school’s use of 
method 


Total Possible Score for Component 1 


2. Comprehensive Design: 
2.1 Existence of written design or plan: name it and give its date 
Name: 
Date: 
2.2 Contents of plan (yes/no to each item): 
2.2.1 Inclusion of needs assessment or other performance data; 
2.2.2 Reference to specific financial resources 
2.2.3 Indication of strategic use of financial resources 
2.2.4 Statement of quantitative performance goals 
2.2.5 Discussion of specific curricula 
2.2.6 Discussion of assessment tools 
2.2.7 Discussion of professional development 


2.3 Breadth of plan in covering all school operations (including, implicitly, all other 
CSRD components) (high, medium, low, defined as follow): 


high: covers all CSRD components (whether implicitly or explicitly) 
medium: covers four or five components, but not all 
low: covers one to three components only (also name them) 


a Total Possible Score for Component 2 


* yes=1 and no=0 


PRP PrP PPP 


(Continued) 
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Appendix E (Continued) 


Component Measure Score* 
3. Professional Development: 
3.1 Strong content focus [???]: yes no iE 
3.2 Range of PD days required or taken by average teacher per year: 7+ 4-6 1-3 7+ =3 
4-6=2 
1-3=1 
3.3 Evidence that preceding estimate excludes traditional teacher set-up (in yes no |Make part 
the fall) and teacher clean-up (in the spring) days: of 3.2 total 
3.4 Evidence of collective participation of groups of teachers from the same yes no 1 
school 
3.5 Evidence of some PD taking place in the teacher’s classroom—e.g., yes no 1 
mentoring 
3.6 Explicit guidance to align PD with standards, curriculum, or assessment yes no 1 
tools 
Total Possible Score for Component 3 7 
4. Measurable Goals and Benchmarks: 
4.1 Number of academic subjects covered: No.: 4+ =3 
2-3 =2 
0-1 =1 
4.2 Number of grades covered and total no. of grades in the school: No.:__—s—- No.:____  0.0- 1.0 (%) 
Total Possible Score for Component 4 4 
5. Support within the school: 
5.1 Existence of formal faculty votes on reform or research-based method yes no dl 
5.2 Formal faculty vote(s) on reform or research based method show 75% yes no 1 
support 
5.3 Interviewees voice strong support or enthusiasm yes no 1 
5.4 Two or more interviewees voice dissent or indicate lack of use yes no 
Total Possible Score for Component 5 4 
* yes=1 and no=0 
(Continued) 
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Component Measure Score* 


6. Parent and Community Involvement 
6.1 Emergence of new forms of parent involvement during CSRD years: 
6.1.1 Special parent events 


6.1.2 Programs or opportunities for parents in instructional roles 3-4=1 
6.1.3 Parent advisory or other committees 0-2= 
6.2 Level of parental involvement (high, medium, or low, defined as 
follows): 
high: you’ve observed parents in the school and interviewees voice strong or High high=2 
satisfactory level of parental involvement in school activities 
medium: school get traditional level of parental involvement (e.g., 10% medium medium=1 
attendance) 
low: no evidence of parental involvement beyond a handful of parents and low low=0 


interviewees voice low levels of participation 


6.3 Evidence of at leave least one community organization and one 
school/community event or program 


Total Possible Score for Component 6 4 


7. External Technical Support and Assistance 
7.1 Developer support and assistance (high, medium, or low, defined as follows): 
high: all 3 CSRD years plus during year after CSRD 


medium: at least two of these four years high high=3 
low: one or none of these four years medium medium=2 
low low=1 


7.2 Other external (but non-district) support and assistance 
yes: evidence for a specific source and function on two or more occasions yes no 1 


no: no such evidence (evidence can be documentation, interviewee mentions, 
or direct observation) 


Total Possible Score for Component 7 4 


* yes=1 and no=0 
(Continued) 
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Appendix E (Continued) 


Component Measure Score* 
8. Evaluation Strategies: 
8.1 Existence of a written evaluation plan yes no 1 
8.2 Evidence of written evaluation findings (could even be a memo) yes no 


Total Possible Score for Component 8 
9. Coordination of Resources 


9.1 Evidence of some coordination of funds from different external (e.g. yes no 1 
federal) sources 
9.2 Evidence of some coordination of external and local funds (i.e. core yes no 1 
building) 
Total Possible Score for Component 9 2 
Total 47 


* yes=1 and no=0 
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Corroboratory Assessment of CSRD Implementation 
For 18 Schools in Field-Focused Study 


Appendix F 


CORROBORATORY ASSESSMENT OF CSRD IMPLEMENTATION 
FOR 18 SCHOOLS IN FIELD-FOCUSED STUDY 


Lowest Group (little or no central method or other signs of CSRD) 


School R: Did not implement CSRD during 1998-2001 period, during which school 
had three different principals (a fourth principal ended service in 1997-98, after having 
served for four years and supported the initial reform efforts). School R had started reform 
activities in 1997-98 (prior to CSRD) and then implemented a weak version of CSRD 
during 2001-2002. Thus, during assessment period, had virtually no dosage. 


School E: Had implemented Coalition of Essential Schools in 1998-99, but by 2001- 
02, staff were devoting more time to other methods. School was originally attached to a 
middle school, then separated in 2001-02 and asked by the board to operate as a charter 
school (school had recommended that it close). The school has been pre-occupied by 
restructuring and appears not to have had any vision for comprehensive reform. 
Professional development is limited and disconnected, and there has been no external 
support. 


Low-Middle Group (CSRD dosage limited to some grades, some years, 
and some resistance) 


School M: Adopted Lightspan in its 3rd grade for 1999-2000 and the 3rd-5th grades 
for 2000-01 (first two CSRD years). However, the method was not used throughout these 
grades (with some evidence of strong resistance by some senior teachers), and there was 
little evidence of any use of Lightspan during 2001-02. There were few other signs of 
CSRD reform besides Lightspan, and the original principal abruptly left the school in the 
spring of 2001. 


School I: Began implementing Lightspan in 1997-98, one year before CSRD. By 
2001-02, the most intensive implementation was in the 1st grade (the school had expanded 
from K-3 to K-6 that year). The school had two schoolwide plans, neither offering a 
comprehensive picture or design. Teachers did not share a unified vision, describing a 
“split focus” and a mission that was “way too broad.” The school also has been using other 
reform methods, especially in literacy, but has been negatively affected by district and state 
budget cuts 


School O: Voted to adopt Co-NECT, but district encouraged expansion of SIM, 
which had started in 1996-97 (prior to CSRD) because of the school’s low performance on 
state accountability tests in 1996-97 (performance had improved by 2000-01). The school 
improvement plan provides a unifying plan of action, but CSRD may be a program, not a 
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catalyst for reform. In 2000-01, school also transitioned from SFA (adopted in 1996-97) to 
district’s reading curriculum. SIM use is uneven and declines in weeks prior to state tests. 


[O is not ranked higher because of the multiplicity of methods, either considered or 
implemented, the noncentrality of CSRD, and the unevenness of SIM implementation. ] 


Middle Group (CSRD implemented, but unevenly or without strong staff or district 
support) 


School K: Designated by state as in need of improvement in March 1998 and 
therefore required to adopt a proven method, the School started SFA in 1999-2000. SFA is 
in place, but faculty in grades 4-6 have complained about SFA’s rigidity and voiced 
pleasure at cancellations of 90-minute block. School reform plan addresses all CSRD 
components. The school started as a K-3 school in 1999-2000 but after renovation became 
a K-6 school by 2001-02. 


School N: Based on district initiative, 98 percent of faculty voted to adopt SFA, 
which started in 1998-99, with full implementation during the next two CSRD years. 
Teachers concerned about continuing SFA in 2002-03 because of state accountability 
system (e.g,, teachers are accountable for homeroom students even though they may be 
regrouped under SFA). In 2001, district designed new format for school improvement 
plan, covering goals but only citing strategies in a general manner. 


School B: Started implementing reform a year before CSRD (1998-99) and then 
vigorously pursued comprehensive method (HSTW) during the next two years. However, 
by spring 2001 support for reform appeared to be fading, with faculty voting to avoid shift 
to block scheduling (a key feature of HSTW)—possibly because of a state report showing 
that non-block schools had slightly larger increases in verbal SAT scores. District’s plan to 
merge two high schools for 2003-04, with School B’s principal to be the principal of the 
merged school. 


School H: Voted to adopt SFA in 1997-98 and implemented it in 1998-99 (the first 
CSRD year); by 2000-01, all classrooms were using SFA, and the developer rated fidelity 
high. The school also had a comprehensive school improvement plan that appeared to be 
widely understood and aligned with CSRD’s components. However, the staff decided to 
stop using SFA in Nov. 2001, reflecting lack of district support and concern that the school 
could perform poorly on district and state assessments by not using the district’s reading 
curriculum. 

[H could be ranked lower because the initial adoption was related to a district, not 
school initiative, and because of the transition away from SFA.] 


School J: Over 80 percent of faculty voted to implement Performance Tasks, a 


locally developed method. By CSRD’s third year (2001-02), PT was being implemented to 
some extent in all classrooms, but exact use of method and tasks depend on teacher and 


F-2 


academic subject. School has a core steering committee managing and integrating all 
reform components, and district staff also has been supportive. However, principal who led 
all of the school’s efforts retired in June 2002. 

[J is not ranked higher because of the lack of rigor in implementing the method and 
the thin-ness with which the tasks cover certain subjects—e.g., mathematics. ] 


High-Middle Group (CSRD implemented with full staff support and district 
alignment, at least during first two or even three CSRD years; some conflicts with 
other priorities) 


School C: Implemented Co-NECT in 1999-00, based on unanimous faculty vote. 
School had adopted Accelerated Schools 12 years earlier and already had a participatory 
management structure and an orientation toward reform. School Improvement Plan (SIP) 
is aligned with district and state goals and standards. The plan embraces all funding 
sources as part of an integrated plan. Strong principal support, but some teachers fear that 
Co-NECT competes with their efforts to raise achievement scores, and teacher turnover has 
been high. 


School D: Positive experience with SFA starting in 1997-98 led to choosing and 
implementing MathWings in 1999-00, with over 80 percent of faculty voting support. 
District’s strategic plan and requirement for school improvement dovetails with CSRD’s 
comprehensiveness, although district was promoting its own curriculum in mathematics 
and reading, and the school could not take advantage of district-supported professional 
development. Alignment of SFA and MW methods with state’s standards and assessments 
may be questioned. 

[D is ranked higher than C because of strong district planning and principal support. ] 


Highest Group (CSRD implemented with full and ongoing staff and support and 
district alignment) 


School L: Started implementing Behavioral Modification, a locally developed 
method, in 1999-2000, with support from principal and school leadership team. By the 
third year, possibly 70 percent of staff were using the method. However, principal retired 
in 2000-01, and support for BEH practices may wane. District-mandated strategic plan 
embraced all nine CSRD components, with plan identifying desired activities and not just 
goals. BEH is highly compatible with district’s curriculum preferences. 


School F: Reform efforts had been going on prior to 1998, when 100 percent of 
faculty voted to adopt Coalition of Essential Schools, which started under CSRD in March 
1999. In September 2000, school started two-year award for High Schools That Work, 
implementing block schedule that also was in keeping with CES’s guiding principles. 
School improvement plan embraces all CSRD components, and collaborative groups and 
action teams work together toward a vision for school-wide reform. 
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School G: Was in process of implementing teacher teams, cross-grade planning 
groups, and whole-school professional development prior to CSRD. Enthusiastically 
adopted locally developed literacy model. School’s small size, collaborative management 
style, and history of reform all lead to strong reform support. District-initiated school 
improvement plan has played central role in establishing reform comprehensiveness, with 
faculty members signing plan to indicate their understanding of it. 

[G is ranked higher than F because of strength of alignment with and support from 
district. ] 


School Q: Had been using a district-sponsored literacy instruction since 1995-96, 
showed interest in adopting a comprehensive method in 1997-98 (prior to CSRD), and then 
implemented Accelerated Schools—compatible with ongoing curriculum. School 
improvement plan, school reorganization, and changes in school governance all show the 
comprehensiveness of the reform, with teaching cadres and an integrated standing 
committee, which should make ongoing reform efforts less dependent on any single 
individual. 

[Q is ranked higher than G because of comprehensiveness of AS, compared to locally 
developed literacy model.] 


School P: School implemented High Schools That Work, later complemented with 
Making Schools Work (from the same developer), and linking middle and high school 
reform efforts. Implementation and fidelity both rated high. Only questionable conditions 
involve political and social conflict, presence of three principals, and turnover of 
superintendent during the CSRD award period. In addition, school is to be merged into the 
only other high school in the district, starting in 2003-04, and merger plans and conditions 
have not been clarified. 

[P is ranked higher than Q because of comprehensiveness of HSTW, even compared 
to AS.] 


School A: School began investigating Accelerated Schools in 1996-97, and 97 
percent voted to start implementation in 1998-99. Staff strongly supports reform, showing 
familiarity with CSRD’s components. Teachers are members of cadres, and principal and 
teachers use collaborative problem-solving process and shared decisionmaking believed to 
be central to success. Parent participation has been high. School appears to be an 
advanced AS site and has been recognized by state and CSRD as a model CSRD site. 
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